next »

[J-Snake]

If it's more about a classic game experience for you, I could see advanced AI distracting the hell out of the player

Yeah, but that's because Crysis is broken on some technological and game design level.

or the opposite case where the AI is perfect at everything and the game is unplayable.

That's usually an indication of inherently cheap AI design. Think about the fighter AI example I just provided: it is, by design, inherently impossible to make it perfect because it takes human limitations into account. It means it cannot predict your next move with absolute certainty, and that's where things start to get interesting.

[fluffrabbit]

Yeah, but that's because Crysis is broken on some technological and game design level.

Not necessarily. Human powers of reason are computationally hard. If you assume that Crysis uses a neural network or some kind of emergent structure to control its AI, anything that was overlooked is going to stick out. If a simple state-driven AI were used instead, you would have the opposite problem where the enemies kill you instantly unless they were deliberately designed to be weaker.

That's usually an indication of inherently cheap AI design. Think about the fighter AI example I just provided: it is, by design, inherently impossible to make it perfect because it takes human limitations into account. It means it cannot predict your next move with absolute certainty, and that's where things start to get interesting.

That could totally work with a fighting game because controlling the human body is hard. Not for us because we have a few billion neurons, but give the controls to an agent with only a few thousand neurons and any efficiency gains would likely be overshadowed by limited computing resources. It would probably "feel" fair from a gameplay perspective. You could use NEAT with novelty search or maybe an ultra-optimized objective search method like ES-HyperNEAT. There's a great talk on the challenges

.

[J-Snake]

Human powers of reason are computationally hard.

It doesn't mean that AI in the game has to take an unfeasible amount of computation in order to work. A game has already a defined and very specific context in which the AI operates. Good AI design exploits that, bad or "generic" AI doesn't, a sure way to break the game.

[fluffrabbit]

Human powers of reason are computationally hard.

It doesn't mean that AI in the game has to take an unfeasible amount of computation in order to work. A game has already a defined and very specific context in which the AI operates. Good AI design exploits that, bad or "generic" AI doesn't, a sure way to break the game.

You've got it all wrong. All AI takes the parameters into account. What you're describing is a state machine, which is just the "kill it until it's dead" AI, like in Quake. Any AI that simulates an organism, a brain, or natural processes is going to lead to interesting behavior. Can it break the game? For sure, but so can a human player. This so-called "generic" AI takes several times as long to implement as state-driven AI, but it leads to more emergent behaviors and is the only effective way to get physically based movement and, on some level, actual intelligence.

[J-Snake]

All AI takes the parameters into account.

What was the point of that sentence?

I am not describing any type of technology how to realize AI. Notice that I am talking about something more fundamental. That is the benefits of knowing the context within the AI operates. Using neural networks, or whatever, doesn't make this demand obsolete.

This so-called "generic" AI takes several times as long to implement as state-driven AI

Btw. it is completely irrelevant how long it takes to implement any AI if it isn't superior to simpler variants which exploit context knowledge in a more efficient and robust way.

[fluffrabbit]

All AI takes the parameters into account.

What was the point of that sentence?

I wanted to avoid saying "takes the world into account" because the frame of reference for the AI system is relevant, as you say, but I have a different subjective take on it.

This so-called "generic" AI takes several times as long to implement as state-driven AI

Btw. it is completely irrelevant how long it takes to implement any AI if it isn't superior to simpler variants which exploit context knowledge in a more efficient and robust way.

Right, so, you can pre-train a neural network to repeatably outperform stade-driven AI, like playing back a tape. Lots of games do this, and IMHO it's BORING.

Think about it. We have the technology to put virtual autonomous agents into games that can not only make their own decisions but also surprise us. Sometimes the results can be more fruitful with something like novelty search, but that's the tech side. The subjective side is that you can take a concept like Wobbledogs and really let loose with the AI's freedom. Disregard balance, stats, and all that game stuff. Some people come for the awe of the tech doing its thing, walking around, making decisions, surprising us.

[J-Snake]

I wanted to avoid saying "takes the world into account" because the frame of reference for the AI system is relevant, as you say, but I have a different subjective take on it.

Yeah, but the question is how much does it take the context into account. There can be a big difference between different AIs. For example, Far Cry (2004) takes a lot more context (mostly static geometry, a lot of information can be pre-computed) into account than Crysis.

Right, so, you can pre-train a neural network to repeatably outperform stade-driven AI, like playing back a tape. Lots of games do this, and IMHO it's BORING.

Ok, but I am not implying that. In a (good) game, there is a valid space of actions the AI should be able to perform at a given time, and then there is an invalid space it must not enter. So the purpose of context information is not to castrate the AI, but to detect  invalid space efficiently. Furthermore, efficient utilization of context information can define an efficient (precomputed) data format for the AI to operate on instead of some raw native data. Operating on a different data type space can yield huge performance gains.

Think about it. We have the technology to put virtual autonomous agents into games that can not only make their own decisions but also surprise us.

If you recall the fighter AI example, its single goal is to surprise the player since that's the key to success in a fighter. So yeah, obviously I am interested in interesting AI;)

[fluffrabbit]

Yeah, but the question is how much does it take the context into account. There can be a big difference between different AIs. For example, Far Cry (2004) takes a lot more context (mostly static geometry, a lot of information can be pre-computed) into account than Crysis.

I have never had the pleasure of playing Far Cry. I operate under the assumption that all the nitty gritty details about the world need not be factored into the AI. The more the AI believes it exists in that world, the less it acts like a troll and the less entertaining it is. Obviously there are basics like level geometry, and maybe Crysis only uses simple stuff like waypoints, I haven't looked into it.

Ok, but I am not implying that. In a (good) game, there is a valid space of actions the AI should be able to perform at a given time, and then there is an invalid space it must not enter. So the purpose of context information is not to castrate the AI, but to detect  invalid space efficiently. Furthermore, efficient utilization of context information can define an efficient (precomputed) data format for the AI to operate on instead of some raw native data. Operating on a different data type space can yield huge performance gains.

That's a little over my head. Maybe the state machine overrides the AI when necessary (overtaking in a race, diving for cover), but for the most part I would be ok with the critters wandering off into the desert or whatever; physical constraints are more organic.

If you recall the fighter AI example, its single goal is to surprise the player since that's the key to success in a fighter. So yeah, obviously I am interested in interesting AI;)

That's great. But I think the search space could be bigger than two humans and a floor, like, say, an island with grotesque humanoids learning to ambulate and interact in an entirely organic way.

[gimymblert]

Jsnake is right, it's a design problem, ai technique is largely irrelevant, if your problem is ill define you can't apply the technique.

The problem is that it's always framed as a performance or autonomous discrete agent problem ... it's not, like at all. It's an experience problem, that's why this technique mostly don't caught up, people are solving teh wrong problem.

Also people have a faulty idealistic perception of AI, technically the rubberband items selection in mario kart IS an AI, and it's very effective at what it does. BUT then the basis to understand AI (all type) is based on the realization that most basic AI is randomness, given infinite time, it can map the whole problem domain and find a solution. If you understand that, you understand that sophistication is not what define AI, and you can start applying the right technique to the right problem.

In game most AI talk don't solve the right problem, period.

Just to take a popular example, let's have a dumb design context, flanking (it's not a design goal per see, the question as to what you need flanking, to produce what experience to the player is to be answered first). Most people assume the agent must be smart enough to realize there is flanking routes, ie able to analyze its environement and make decision. Well Flanking is not an agent problem, it's a level design problem, it doesn't matter how smart your agent is, if there is no flanking route in your corridor, he will be like a chicken in a shooting stand, he will appear dumb. Meanwhile, you can have a system that detect where the player is and select flanking routes, then occasionaly, select a dumb agent and make it traverse that routes. Most probably the best solution is to have an AI that select a random point around the player, go there, THEN go to the player, with a curated probalistic chance of doing that among many agents it will seem as if they coordinate themselves.

Ai in game is mostly much closer to improvisational choregraphy than autonomous agent.

[fluffrabbit]

You definitely seem to know about game AI insofar as it specifically pertains to games, but if intelligence is defined as the pursuit of goals, the ability to quantitatively optimize is important. That requires machine learning.

Take for example my experiment with Cave Confectioner:



Sure, doggo looks a little weird, maybe even epileptic, but his optimization function is basically to move as much as possible. I didn't hard-code these movement patterns. The oscillating behavior is a consequence of alternating between 32 different candidate neural networks each frame, part of how the genetic algorithm works, so I consider it a success. When I upgrade doggo to reinforcement learning he won't oscillate so much, and I will consider that even more of a success.

[gimymblert]

A behavior tree is enough for: "the pursuit of goal"
It's effectively quantitatively optimizing, because it's basically a search tree in the domain space of action.

Machine learning is as simple as making an affinity like gauge, the system effectively learn statiscally whatever concept is attach to teh gauge, relative to the repeated stimulation of the concept.

I mean yeah if simulating intelligence is your goal why not, but then it's not NN in game, it's NN with game

[fluffrabbit]

but then it's not NN in game, it's NN with game

Amen, brother. I really had to think about this, and I'm inclined to put NN on the back burner for now.

For one thing, the field of reinforcement learning is introduced without any neural networks at all. NN only comes up about 7 hours into the lecture series I'm watching. NN is a useful function approximator, but it doesn't have to be used.

Perhaps more importantly, gamers don't care, and if I want to do this as a livelihood I'm inclined to follow the money away from these esoteric experiments. As fun as they are, most people probably have more fun playing conventional games. That's fine, and that's our goal here on these forums.

[gimymblert]

IMHO if I were to employ ML/NN in games as an experiment, I would use it for social games like the sims or social aspect in games, to evolve more complex social behavior of a few "hero" characters. Especially by forcing the AI to use a natural language generation to express its state strategically according to balancing the various need and some "drama" aspect. Another aspect is to do the same but with training with example of curated improvisational play with human, with pair of situation sequences and reaction sequences (game behavior and chat log).


I mean killing and strategy are kinda well define and finding optimum don't serve games very well as enjoyment. But maximizing social metric might be fuzzy and ill define enough, while having precise metric to apply. But everything like locomotion or any other traditional stuff, are best left with best practice.

[fluffrabbit]

Best practices for murder and mayhem are to make the AI stupid and let the player win. Best practices for locomotion are machine learning.

I have the least faith in current ML when it comes to text generation. Just look at the recent failures of the Loebner Prize. Until software passes the Turing Test, I don't see how dialogue could be made compelling. That would be amazing.

[gimymblert]

nah best practice for locomotion is still astar with a navmesh. NN is totally overkill for something that wasolve decade ago. Machine learning help with blending AAA scale of animation asset, but it's YAGNI for gameplay, it's just cosmetic.

Anyway, you haven't kept up with the pace of current NLP and NLG and neural net, LOOK for BERT, ELMO and TRANSFORMER architecture, read about semantic encoding in manifold within neural network, etc ...

Here is the thing from IMHO, NLG now fail because the model are trained with statistics and not purpose. The feed tons of text, and let the system discover pattern, and it turns with an attention model for long relationship, that's enough for the model to start understand from scratch what's essentially random data (see performance of bert or open ai transformers experiment) but since they aren't using language for purpose (ie a directed goal with balancing constraint like social context) the language stay pure imitation and wandering, even if latest wandering are super credible (on the scale of the imagination of a small child 4 years old, which is impressive). The thing is that we made huge splash in goal oriented AI, with curiosity architecture, that allow for search in sparse reward domain. The key is how do you create a curriculum that allow the ai to learn speach in similar way but unsupervised?


Loebnetr prize chatbot don't use ML at all, they are glorified overstuff eliza program, with hand made pattern matching in mostly AIML. Aldo passing for a human isn't really a goal for intelligence, reasoning is the goal we are going for. For games good enough is key.



source:
https://eng.uber.com/go-explore/
https://openai.com/blog/better-language-models/

[fluffrabbit]

A* and navmesh are omniscient pathfinding algorithms. While they are the "best" in terms of game theory, machine learning produces more realistic results when it comes to what an entity is supposed to see and not see. As for animating the models, there are ample examples, including Wobbledogs.

Loebnetr prize chatbot don't use ML at all, they are glorified overstuff eliza program, with hand made pattern matching in mostly AIML.

Not just AIML, orthodox AIML with canned responses to specific questions someone might ask. Thankfully the contest structure has changed to allow for more experimentation, though IDK why they need submissions 6 months in advance.

Thanks for the tips on the NLP and Uber stuff. I'm looking forward to reading it in a few hours.

[gimymblert]

A* don't have to be omniscient, you can search only in the realm of the fog of war (ie what seen and what has been seen), just because it used as omniscient, don't mean you have to use it that way. I just forgot the name of the technique where you use a* with uncertain domain. (a bit of googling I found back it's call occupency grid: http://aigamedev.com/open/tutorial/occupancy-grid-prototype/).

Here is the thing IN GAMES, the real problem is communicating teh state of the game to the player, technique like above didn't get anywhere, because if the player isn't omniscient, actual intelligent behavior is just as good as noise or random decision. That was the big lesson HALF LIFE, FEAR and HALO taught all AI designer, it doesn't matter how much your agent are intelligent, it only matter that the player perceived them as such ... and generally that mean artificial barks where they tell you out loud and emote their internal state. That's where the choreography claim come into effect, create legit good hooks to react to event happening in game (like the suidical grunt in halo) with good quip (literally the whole fashion of bantering companion like in the last of us or bioshock infinite) and hey presto you have good ai™.

It seems to me you are discovering AI and you are in the honey moon phase where everything is still magical. Which is good, it's how you get motivated to experiment. I just hope you don't get stuck in well trodden place we had already found out the result (which is why I proposed something less experimented with to slight orient you away of the usual goto).

Also in the honey moon phase, there is this reflex to disdain good old technique that work in their own domain. Thrust me, I don't disdain AIML, it's actually quite a sensible solution and approach, and it's quite sophisticate as the sophistication don't reside in the technique in itself, but in the architecture of a specific chatbot. It's only limitation now is that it has to rely on adhoc human encoding from a domain space (no ML) we don't have real mastery, and surprisingly it's not modeling speech, language parsing or langage generation, it's simply modeling "human experience" (ie the data encoded, not the technique per see).

IMHO we need another technique to interface aiml with, to approximate that data. It's what I call the "dictionary problem" (that is the set of all needed human experience) or in AI it's call the "common sense problem", it's not even a data problem, it's a data context, encoding and retrieval problem. It's not specific to AI, it's an overall human problem, it's the fundamental of human communication problem, ie why we still have racism even with a huge amount of people not supporting racism, because failure to understand the root of an experience don't allow to reason on it, AI like human have tone deafness problem when discussing common sense born from life experience. The range of tone deafness being much bigger in AI, because it is expected for an ai to understand common sense, without having to go through at least 4 years of life, like human do since being born. IMHO that's unreasonable expectation push into AI. IN fact the closest AI to almost past through the Loebner test was a simulation of an hungarian kids with broken english, why? because it simulate the experience gap between the two conversant.

That's why I have been nudging you toward social AI, ie an agent like the sims, with traditional AI, but with a NN layer for hi level decision and PURPOSEFUL communication generation. Because the domain is way smaller that trying to simulate the life experience of going to school and stressing for the future and meeting other people as fidel in IRL. That is if the AI properly evolve within the constrain of the simulation. But even a perfect AI right now will not be as good as human for the reason outline, it will simply have a very different culture because experiencing life very differently.

Also you should read about experiment made with ANGELINA
https://www.rockpapershotgun.com/2012/06/04/the-ai-that-designs-its-own-games/

[fluffrabbit]

I am in the honeymoon phase. It's exciting stuff; lots of ways to optimize the nets, lots of ways to represent data. I don't know of anyone else who combined SELU with neuroevolution besides myself, and I'm so new to it that it feels like there is plenty of room for experimentation.

Angelina is interesting. Reminds of the elephants that paint. Inserting images of politicians into pixel art platformers is pretty juvenile, but impressive considering a home computer is about as powerful as an insect.

The issue of computational complexity is what concerns me. I won't attempt any AI that is supposed to work on the level of primates until processing power is up to the task. Several species are born into the world ready to tackle basic challenges within minutes, and I feel that DRL with compute shaders should be able to do basic locomotion and resource collection from zero. Just to have something smarter than a nematode, and for that intelligence to be authentic rather than hand-crafted, would make for better experiences in humbler indie games. Everything else is a game design problem, and I'm not a gamer.

[diegzumillo]

Jeez, I skip one single email updating this topic and miss out on the whole thing.

I noticed the topic got directed towards AI in games. Like, instead of the usual finite state machine each "enemy" or whatever controlled by its own neural network. In general AI for games are too carefully designed to provide a specific experience for something like neural networks to have a place. It's hard to design something so indirectly. But I can totally see people experimenting with that idea anyway. Of course, the main problem is still computational, because now you have many agents running on individual neural networks.

But I feel there are still potential applications that aren't the intuitive use in AI.

Anyway, just to (partially) contradict my point, I found this game on steam: https://store.steampowered.com/app/396890/Gridworld/ It's an evolution simulator (like swim bots or darwin's pool). It's a grid, each point is either nothing, food, or an individual. The individuals are each controlled by a neural network which gets is created by a genetic code and blablabla evolution etc.