Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411575 Posts in 69386 Topics- by 58444 Members - Latest Member: darkcitien

May 04, 2024, 08:56:33 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)tokenizing, parsing, decision tree and other stuff i don't grasp about scripting
Pages: [1]
Print
Author Topic: tokenizing, parsing, decision tree and other stuff i don't grasp about scripting  (Read 2570 times)
nikki
Level 10
*****


View Profile
« on: March 12, 2010, 05:57:43 AM »

I am trying to get my head around user-definable actions.
I think i need a way of using scripts to make if statements, but i just don't understand the principles.
I am not asking for very specific implemetations, more the abstract way of looking at it.

imagine:

there is  an actor and a sandwich.

the sandwich has the action [eat sandwich] , and if the actor does that it will change its motive [hunger].
Now imagine a script language in wich you set this specific action, and make it a little bit more complex. (lets add an extra if , lets say the sandwich is only eatable during the afternoon)

Quickly the code will grow into nested if's , decision tree like.
But how to write such a thing in a sript? (and the real question; how to generate such a script from a few user-inputs (connecting boxes, filling in variables)?)


I believe the question  has to do with tokenizing, parsing and other stuff i don't grasp  Facepalm ,
Could someone shed his/hers light on this?



« Last Edit: March 12, 2010, 06:00:55 AM by nikki » Logged
Movius
Guest
« Reply #1 on: March 12, 2010, 08:04:50 AM »

tokenizing is basically splitting up a string into components parts (tokens). eg. tokenizing "2 + 2" with space as a delimiter would give you the tokers "2","+","2".

Parsing is a more general term for making sense of a string and turning it into useful info. eg. with "2 + 2" this would involve something like tokenizing the string as before and then pushing the resulting tokens onto the processing stack.

There are many solutions for your more general problem. But these are a good place to start.
Logged
Kunal
Level 1
*


is feeling Bit.Core.Trippy


View Profile WWW
« Reply #2 on: March 12, 2010, 08:16:26 AM »

To add to Movius, you might also want to look at this

http://en.wikipedia.org/wiki/Lexical_analysis

There are tools for doing this sort of stuff, like JavaCC,ANTLR etc. but I don't have any experience in using them.



Logged

Kekskiller
Guest
« Reply #3 on: March 12, 2010, 08:39:51 AM »

Writing a whole scripting language is something you do once for your engine, taking a bit of time and necessarily a lot of thinking if you never did it before. A rather... "cheap" way is to use tables like Diablo II did it. Doing this you would have dozens of tables with premade conditions of an object (which you can define in your program for example) and actions by another object (like the player). An action y depending on condition/state x will result in a new state (in cell xy). Moreover if you're not afraid of memory wasting, you can create multiple dimensions of these tables. Possible dimensions/axis of the table could consist of grammar terms like in english: Subject (interacting object), Verb (action by subject), adjective (how is it performed), Object (object that will change it's state). I did a little game with it (or atleast the very basic structure) and it's time-consuming to do. But: It forces you to handle every possible case of interaction and object state. It's also simple to calculate, you don't need to hardcode stuff and if your system is good enough you enable players to mod your game.
Logged
Movius
Guest
« Reply #4 on: March 12, 2010, 08:53:03 AM »

If you absolutely have to use a scripting language for this. I recommend just plugging in something like lua to do the job for you. This is a relatively simple and painless task, but I would recommend limiting it's use.

Search back 2 years or so, every other thread here would be about how the poster was going to implement lua scripting in their game. Should be some tips in there.
Logged
Chromanoid
Level 10
*****



View Profile
« Reply #5 on: March 12, 2010, 09:11:38 AM »

if you only want a fast and simple solution i would use xml.
just find a good xml parser for your programming language and do something like this:
Code:
<object id="sandwich1" text="sandwich" parent="food">
  <action id="eat" text="Eat the Sandwich" icon="eat.png">
    <condition id="active" type="or"><!-- simply ORs all childconditions in the element -->
      <condition type="equals"><!-- returns true when all children are equal -->
        <variable name="global.timeofday"/><!-- returns global game variable time of day (afternoon=4) -->
        <int value="4"/>
      </condition>
    </condition>
    <operation id="execute" type="sequence"><!-- simply executes all children -->
      <mathop type="set" target="player.hunger"><!-- simply sets target-variable  to value of first child -->
        <int value="0"/>
      </mathop>
    </operation>
  </action>
  <!-- ...and so on... -->
</object>

<!-- actions are shown to the user if the childobject with id "active" is true.
if the user wants to execute an action the childobject with id "execute" is executed.
an object shows all its active actions to the user... all xml elements have a class-
counterpart in your code. -->
with such stuff you can implement many things without thinking about stack positions, tokenizing etc... it may become a bit unhandy if you want to script really complicated stuff but it is enough for a fast and stable externalization of game logic...
« Last Edit: March 12, 2010, 09:21:28 AM by Chromanoid » Logged
Zaratustra
Level 7
**



View Profile WWW
« Reply #6 on: March 12, 2010, 12:06:42 PM »

Welcome to the wonderful world of COMPUTER SCIENCE!

Here's how you do go about writing your own compiler:

1) Don't do it.
2) For god sake don't.
3) Learn Lua or python or Ruby integration ANYTHING before you try to harness lex and yacc to your will YOU WILL BE SORRY.
Logged

shrimp
Level 5
*****


View Profile WWW
« Reply #7 on: March 12, 2010, 12:23:56 PM »

Whilst it's true that one should use Lua or whatever instead of rolling ones own scripting language, it's not really what nikki is asking. (I think)

Quote
(and the real question; how to generate such a script from a few user-inputs (connecting boxes, filling in variables)?)

You could certainly make the output of your graphical flowchart Lua or Python, but how do you generate the script from the diagram?

I wonder if a script is not the answer anyway. Maybe you'd be better off having [insert OOP language] classes for each type of node in the flowchart and references to other instances of classes for the links between them.

As a hard-coded example:

Code:
Node n1 = new IfNode(val1, val2, Comparison::LessThan, thingToProcessIfTrue, thingToProcessIfFalse);
Node thingToProcessIfTrue = new PrintMessageNode("val1 was less than val2!");
Node thingToProcessIfFalse = null; // got bored ;)

Not sure how workable this is.
Logged

Crimsontide
Level 5
*****


View Profile
« Reply #8 on: March 12, 2010, 01:37:22 PM »

Welcome to the wonderful world of COMPUTER SCIENCE!

Here's how you do go about writing your own compiler:

1) Don't do it.
2) For god sake don't.
3) Learn Lua or python or Ruby integration ANYTHING before you try to harness lex and yacc to your will YOU WILL BE SORRY.

I wrote my own compiler...

Btw, I'd suggest you use the spirit parser.  All the fun and like 1/8 the work.  No really its actually pretty sweet for small stuff.
Logged
lansing
Level 2
**


View Profile
« Reply #9 on: March 12, 2010, 03:29:20 PM »

Welcome to the wonderful world of COMPUTER SCIENCE!

Here's how you do go about writing your own compiler:

1) Don't do it.
2) For god sake don't.
3) Learn Lua or python or Ruby integration ANYTHING before you try to harness lex and yacc to your will YOU WILL BE SORRY.

Wise words.
Logged
David Pittman
Level 2
**


MAEK GAEM


View Profile WWW
« Reply #10 on: March 12, 2010, 03:41:03 PM »

lex and yacc are the easy way out. Real programmers write entire compilers from scratch.

But seriously, as someone who has written an entire (relatively simple) compiler from scratch: It's not that hard, and it's actually a lot of fun, but it's probably not necessary.

To the original question, I'd say you're already on the right track by referring to it as a decision tree. You don't necessarily have to get into script compilation. It sounds like you want a visual interface and you've made the assumption that the elements in the visual interface needs to be transformed into a script before they can be "executed" by your program. I disagree. If you're already explicitly building a tree of operators and variables, then converting into script is actually taking a step backwards, adding a layer of abstraction to the logic.
Logged

nikki
Level 10
*****


View Profile
« Reply #11 on: March 12, 2010, 06:10:21 PM »

thanks for all the advices!
i went to the bookstore today and had a long look at this book

To the original question, I'd say you're already on the right track by referring to it as a decision tree. You don't necessarily have to get into script compilation. It sounds like you want a visual interface and you've made the assumption that the elements in the visual interface needs to be transformed into a script before they can be "executed" by your program. I disagree. If you're already explicitly building a tree of operators and variables, then converting into script is actually taking a step backwards, adding a layer of abstraction to the logic.


i think your completely right about that assumption i made,
The decision tree will be my friend! , i wrote a lititle test prgram that builds a tree with decisions and possible actions, all decisions are extended classes so all the logic i need integer/float( ==, <, >, <> ) is actually a specific extended class that take some parameters, translating that to script and then that script back into the same code is not necessary Smiley

thanks for all  the tips and links peopel, some reading to do!
« Last Edit: March 12, 2010, 06:17:23 PM by nikki » Logged
st33d
Guest
« Reply #12 on: March 13, 2010, 03:10:33 AM »

Fucking hell!

That's just the book I need right now. I've started on the task of coding complex AI for my game.

*orders on Amazon
Logged
st33d
Guest
« Reply #13 on: March 16, 2010, 01:47:09 AM »

Whoo!

It came through the post today! Just as I've started work on minion AI in my game!
Logged
nikki
Level 10
*****


View Profile
« Reply #14 on: March 16, 2010, 04:45:02 AM »

Cool!, I bought i too!.

A new more specific question:

I've been creating a program to build decision trees, now I want to save them.
I figured breadth-first search would be the proper solution to get all nodes in the proper order.



Code:
start
decision
decision
action
action
action
decision
end
end
action
action
end
end

Now i need to get my head around a way of saving the nodes to the harddisk .
only the action nodes (1 child), and decision nodes (2 children) actually matter.
I am guessing i need a temporary ID for the nodes, so at loading i am able to put them where they have to go.

Anybody can tell me the way to properly do it?
(ps when i search for node saving graph etc i keep on finding JAVA libraries that are pretty high-logic)

« Last Edit: March 16, 2010, 04:49:09 AM by nikki » Logged
skyy
Level 2
**


[ SkyWhy ]


View Profile
« Reply #15 on: March 16, 2010, 04:59:36 AM »

Have you seen this? Doing your own tools is fun, but I decided to use this because at times... I just cannot be arsed Wink
Logged

Chromanoid
Level 10
*****



View Profile
« Reply #16 on: March 16, 2010, 05:13:07 AM »

make a list with all nodes you want to save, replace references to each other with the positions in the array. save the array to disk.

or (when each node is referenced only one time)
begin with the root and save its data (including a childcount) to the file then do this recursivly with the children and so on. when reading it in you just have to use the same recursive way reading the children in...
pseudocode:
Code:
writeNode(Node n, File f){
 f.writeString(n.name);
 //more data
 f.writeInt(n.children.length)
 for(Node c in n.children){
   writeNode(c,f);
 }
}
Node readNode(File f){
 Node n=new Node();
 n.name=f.readString();
 //more data
 int childCnt=f.readInt();
 n.children=new Node[childCnt];
 for(int i=0;i<childCnt;i++){
  n.children[i]=readNode(f);
 }
}
Logged
Overkill
Level 3
***


Andrew G. Crowell


View Profile WWW
« Reply #17 on: March 16, 2010, 10:28:27 AM »

Generally, language creation is a bad idea. If you can avoid writing a parser from scratch or reinventing a language, it's probably good for your sanity. However, sometimes you can't shake the urges and want to get your hands dirty with parsing.

For language building, I would seriously consider ANTLR. It generates LL(*) parsers, from a unified grammar that can contain both nonterminal and terminal definitions, and it has built-in semantics for building abstract syntax trees (with markers to omit or promote nodes in the tree as needed) without having to put in special rule user-code. It also has a tree parser built-in, which might be handy. Another nice thing is with ANTLR, you can actually specify rules for lists-of-things as iterative "regular expression"-like constructs, instead of writing recursive grammar rules, making it a little less time consuming to write complicated things.

Alternatively, you COULD use Lex and Yacc, but it is painful to write and debug, and does not come with a tree parser (though you could hook in treecc or similar).



I'd discourage writing a compiler completely from scratch without existing tools, but it's doable. First write a lexical scanner to convert input into tokens, then feed those to your syntax parser (probably of recursive-descent LL family, since they're much simpler to code by hand). Should basically look ahead at a sequence of tokens (maybe LL(1)) and decide which branch of your parser to follow or terminate based on your current state.

If you compute first and follow sets of a formal BNF notation you could ensure your grammar doesn't have ambiguities, but these are moderately painful.


If you're doing this from a game's built-in editor and selecting symbols from a menu instead of actually having the user type things in, you could just generate the AST or opcodes directly, avoiding parsing entirely.

Logged

nikki
Level 10
*****


View Profile
« Reply #18 on: March 17, 2010, 06:01:41 AM »

@skyy : that seems like a very good solution, however i have my mind set on building my own this time. I could steal some ideas from that library though..

Now I am moving my "interaction decision-tree maker" into my world-editor.
The decision tree has 2 nodes (action with 1 childnode, to run when the action is finished) and decision nodes( with 2 child nodes, 1 for false, 1 for true)

The actual decisions are either Boolean, integer in range, integer in set, float in range, float in set.

The actions are functions with possibly more then 0 parameters.

Since the logic is object logic to be used by actors. The files are saved in the object..
The object also keeps reference to the actor that its interacting with and the world that its in.

So decisions and actions can acces member-values of the actor and the world.


I don't know why i am writing this, i guess its to say it out loud!

Logged
Pages: [1]
Print
Jump to:  

Theme orange-lt created by panic