Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1369580 Posts in 64364 Topics- by 56384 Members - Latest Member: multimedia.capitalworks

November 22, 2019, 12:25:51 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Neural networks in games
Pages: [1] 2
Print
Author Topic: Neural networks in games  (Read 1624 times)
diegzumillo
Level 10
*****


This avatar is so old I still have a some hair


View Profile WWW
« on: May 14, 2019, 04:59:52 AM »

Neural networks. Possibly the most trendy of all tech buzz words. I mean, I can totally see the hype! it's cool as hell. Even I decided to get on board and learn some of the basics. I made the classic python code to that identifies numbers written by hand in small images. The hello world of neural networks. But that's as far as I got. Every idea I can possibly have with this thing would need a more flexible environment than python.

Which brings me to this topic. I am finding the weirdest thing in the world that we are not getting flooded by games using neural networks in one way or another. Maybe it is being used behind the curtains and I'm just not aware of, like path finding or some fancy AI. But no one is putting it on our faces, like a selling point. I guess it might be hard to find a good use where performance is not an issue.

The most obvious idea: I wanted to make an arena type of game, where you battle bots controlled by neural networks evolved with a genetic algorithm. You defeat 10 bots, the next batch of bots is created based on the best of the previous batch, using a genetic algorithm to evolve the neural network of each bot. Maybe add a layer of narrative on top of that, like you need to traverse a long stretch of land, like desert, of a skynet dominated kind of post apocalyptic world. And new bots are constantly being generated and chasing you. The more you fight the better the AI gets... I like that. Encourages stealth a little bit. But maybe fighting them gifts you with better gear.

Logged

fluffrabbit
Guest
« Reply #1 on: May 14, 2019, 09:50:45 AM »

I don't speak for everyone, but I find machine learning to be an incredibly difficult field to understand. Graphics programming is easy because you get to look at pictures along the way, but not so much with ML. The end result is photorealistic yet braindead games.

From what I have been able to research, the class of algorithms you need is known as neuroevolution, not to be confused with neural networks. A neural network is baked in advance like a lightmap (to use a graphics term) while neuroevolution is more like dynamic lighting, and adapts better to a player's changing behavior patterns because it constantly re-organizes itself.

A good start would be HyperNEAT and related algorithms. These are university research projects, not proper game projects, so there is a bit of a learning curve if you want to integrate the algorithms into your own projects.
Logged
BorisTheBrave
Level 10
*****


View Profile WWW
« Reply #2 on: May 14, 2019, 12:42:33 PM »

Training ML nowadays is pretty slow and requires masses of data. Not a good fit for games. But pre-trained ML is starting to make an appearance.



Logged
fluffrabbit
Guest
« Reply #3 on: May 14, 2019, 01:22:58 PM »

That's pretty impressive. For a large problem space like character animation, pre-trained neural networks may be useful for their efficiency. What I had in mind was more along the lines of high-level AI strategy decisions where there isn't as much to figure out.

Neuroevolution has been used as a learning process for robot locomotion, which I think is more interesting in terms of game mechanics. Sorry I can't find a video.
Logged
diegzumillo
Level 10
*****


This avatar is so old I still have a some hair


View Profile WWW
« Reply #4 on: May 14, 2019, 01:27:14 PM »

Training ML nowadays is pretty slow and requires masses of data. Not a good fit for games. But pre-trained ML is starting to make an appearance.





I've seen that. It's pretty cool! That's kind of what I was thinking there should be more around. Pre-trained NNs for specific tasks.
Logged

fluffrabbit
Guest
« Reply #5 on: May 14, 2019, 04:57:58 PM »

I dunno. "Specific tasks" like stepping over rocks isn't the most mind-blowing use of AI. I thought you were envisioning a vibrant world full of hopping and tumbling monsters that figure out their own bodies through trial and error.
Logged
diegzumillo
Level 10
*****


This avatar is so old I still have a some hair


View Profile WWW
« Reply #6 on: May 14, 2019, 05:15:47 PM »

I dunno. "Specific tasks" like stepping over rocks isn't the most mind-blowing use of AI. I thought you were envisioning a vibrant world full of hopping and tumbling monsters that figure out their own bodies through trial and error.

You extrapolated a lot from that quick idea I thrown out in the first post lol. I mean, that sounds awesome! but I suspect something like that isn't within the realm of possibility yet. It might be doable if each bot has, like, 4 degrees of freedom (4 inputs). Put it on a plane (x,y) position, angle, and shooting. Then generating new bots every time you kill 10 shouldn't lag the game more than a few seconds which you can put in a loading screen or whatever. It depends on the number of layers and it would take some trial and error to see how many it takes to see some interesting emerging behavior.

But don't get caught up in "my idea"! this is not a "help me do this" kind of topic. I am genuinely curious to know why neural networks aren't yet being used extensively in games and I shared my idea because I think it sounds neat and it's a outrageous example of the topic.
Logged

fluffrabbit
Guest
« Reply #7 on: May 14, 2019, 08:54:11 PM »

It's been done as research demos, but with HyperNEAT and rtNEAT, not neural networks. The critterding package in Debian has a very primitive example of evolutionary algorithms, no NEAT AFAIK. Search around on GitHub for more advanced stuff.
Logged
Qwerku
Level 0
**


View Profile
« Reply #8 on: May 15, 2019, 04:05:46 AM »

I used to program neural networks for control systems, and the biggest bottleneck I had was computational power for the amount of time I had for each control loop. This problem has only gotten worse now that neural networks have a ridiculous amount of layers and interconnections.  Think about how much computational power OpenAI needed to play one game of DotA2!

You also need to realize that the larger the neural net, the more training it needs. However, it also needs a very diverse training set, or it can become overtrained on a small subset of what it will actually experience "in the wild," and that leads to terrible results.

We simply don't have the need for this kind of complexity in games right now, and it would easily consume an entire GPU to operate a pre-trained, real-time neural net with any sophistication.

Remember, even with scripted AI, the AI can become too "clever" and outsmart players in FPS games. The players don't realize this and think the AI is cheating. If anything, we've had to dumb game AI down more than scale it up.

I think AI needs to have a revolutionary leap beyond dumb-simple neural net architectures where it can learn on the fly like humans, and games need a revolutionary leap in social interactions between entities before we will see the magic of these two worlds combining (that's where most of human intelligence is centered around, after all). At this rate, I think this will probably start happening in the latter half of the 2020s with some bad prototypes spawning in a few years from now.
Logged
Daid
Level 3
***



View Profile
« Reply #9 on: May 16, 2019, 12:18:59 AM »

Remember, even with scripted AI, the AI can become too "clever" and outsmart players in FPS games. The players don't realize this and think the AI is cheating. If anything, we've had to dumb game AI down more than scale it up.
I think the whole reason why we are not seeing this used in games boils down to this. As perfect enemies do not make for a fun experience.
Same holds up quite well for randomness in enemy AI as well. Too much randomness and enemies become unpredictable and impossible to fight in a fun way. People enjoy learning the system to beat it. If the system doesn't exist (too much random) or evolves faster then the player (to smart AI) then this joy is lost.


Doesn't mean there are uses for machine learning in game development. But I think the use is more at the "prepare data" phase. For example train your AI to play your RTS to see if there is a single "best tactic" of only building a single unit type.
Logged

Software engineer by trade. Game development by hobby.
The Tribute Of Legends Devlog Co-op zelda.
EmptyEpsilon Free Co-op multiplayer spaceship simulator
BobbyJones
Level 0
*


View Profile
« Reply #10 on: May 23, 2019, 12:49:36 AM »

I think one cool way to use ML in games is to analyze procedurally generated data. For example terrain generation. It will be able to tell you if the terrain is realistic. That is how I would use it anyways. I also would use it as part of an in-game AI for MMOs but not in the usual way. I would make it where users can build/purchase a bot but the bot must be scripted to use some ML algo and must be trained over time. Which would add an interesting dynamic to MMOs I feel.


Edit: Oh and one more thing that is pretty far out in terms of practicalness but earlier today I thought of a predictive system to combat latency. Basically, if there is a severe amount of latency the server can use ML to predict what each player is likely to do. Then send that predicted action to clients. Correct if wrong. Since we will know if it is wrong we can train the system over time. Tailored to each individual player. I think this would be extremely cool if it works.

Edit2: it would be like branch prediction in CPUs.
« Last Edit: May 23, 2019, 12:57:31 AM by BobbyJones » Logged
qMopey
Level 5
*****


View Profile WWW
« Reply #11 on: May 23, 2019, 01:45:57 PM »

I've always liked the idea of using ML to get "fuzzy" results for "qualitative" questions, and then use that result + a threshold inside of hand-written AI code to determine what to do.

For example say an AI is playing an RTS and wants to know whether or not a group of its own units, who are fighting, should run away or continue fighting. It would be possible to generate lots and lots of controlled scenarios where the answer to run or not is known a-priori. Then run a training algorithm to output a fuzzy answer from 0 to 1. 0 meaning stay, 1 meaning run, and in the middle meaning mostly unsure.

A hand-written higher level algorithm can query the net and get a retreat result. If the result is above X, the ai can transition into a retreating state. The threshold X itself can even be adjusted during run-time (or just hardcoded).

This idea can work in many games, not just RTS games. The concept of "keeping fighting" vs "run away" is common in many games. Even pokemon style games in deciding to retreat or not, based on inventory, hp, status ailment, etc.

And so whether or not the AI plays perfectly is not relevant -- the ML itself can generate a useful metric. The higher level code that consumes the ML metric can implement "imperfection" just fine, often times in trivial ways.
Logged
kason.xiv
Level 0
**


View Profile
« Reply #12 on: June 09, 2019, 04:42:27 PM »

Every idea I can possibly have with this thing would need a more flexible environment than python.

Most companies I personally know that are working with ML use python - no offense, but if it's good enough for them it's probably good enough for you Tongue  Shrug
Logged
fluffrabbit
Guest
« Reply #13 on: June 09, 2019, 05:05:43 PM »

Every idea I can possibly have with this thing would need a more flexible environment than python.

Most companies I personally know that are working with ML use python - no offense, but if it's good enough for them it's probably good enough for you Tongue  Shrug

True. Tensorflow was huge a couple years ago, and that's Python-based. However, I think it's important to understand why.

Python is a popular language among non-programmers, which is a group that includes university researchers. Lots of math and physics folks use Python because it's a tool "for them". Not to dis Python, it's got a nice syntax, but the Python runtime and ecosystem are not well-suited for games. You can do AI research in Python and rely on toolkits that do "heavy lifting" with low-level code. It's mostly going to be GPU-based anyways for modern "big data" uses, and GPU programming is a specialist skill that the "heavy lifting" libraries take care of.
Logged
J-Snake
Level 10
*****


A fool with a tool is still a fool.


View Profile WWW
« Reply #14 on: June 10, 2019, 02:47:50 PM »

I am finding the weirdest thing in the world that we are not getting flooded by games using neural networks in one way or another. Maybe it is being used behind the curtains and I'm just not aware of, like path finding or some fancy AI. But no one is putting it on our faces, like a selling point.
You first need to become/have a good designer in order to make good AI work. AI itself doesn't make a game work. And good designers are rare.

But sophisticated AI can be an interesting ingredient for many games, that's for sure. For example, I would like to see it in fighters, operating within a realistic constraint model reflecting human limitations (like reaction time and execution speed). That way, it could be a thrill to play even against AI and see how it competes with human mind games.
Logged

Independent game developer with an elaborate focus on interesting gameplay, rewarding depth of play and technical quality.<br /><br />Trap Them: http://store.steampowered.com/app/375930
fluffrabbit
Guest
« Reply #15 on: June 10, 2019, 03:09:18 PM »

Quote
You first need to become/have a good designer in order to make good AI work.
Regretfully, that is the case with conventional techniques. Most neural networks have a fixed size and need manual tweaking and tuning. However, there are some more modern forms of neuroevolution and reinforcement learning emerging that are supposed to make that fiddling a thing of the past, given enough computing power.
Logged
J-Snake
Level 10
*****


A fool with a tool is still a fool.


View Profile WWW
« Reply #16 on: June 10, 2019, 03:30:23 PM »

That may be true for certain cases. But in general, you still need good game design first.
Logged

Independent game developer with an elaborate focus on interesting gameplay, rewarding depth of play and technical quality.<br /><br />Trap Them: http://store.steampowered.com/app/375930
fluffrabbit
Guest
« Reply #17 on: June 10, 2019, 03:42:18 PM »

That may be true for certain cases. But in general, you still need good game design first.
For sure. A purely AI-driven game is a tech demo. Still, the technologies themselves are often a bigger draw for me than the gameplay. For instance, have you seen Wobbledogs?
Logged
J-Snake
Level 10
*****


A fool with a tool is still a fool.


View Profile WWW
« Reply #18 on: June 10, 2019, 04:22:13 PM »

Still, the technologies themselves are often a bigger draw for me than the gameplay.
Yeah, that's a different story then. In games, I don't care that much about technologies if the gameplay isn't right. To satisfy my personal standards, getting gameplay related tech and design right end-to-end is a more relevant challenge than toying around with new concepts/technologies to teach dogs how to walk etc. (it can certainly be interesting in its own right, but I have other preferences).

Logged

Independent game developer with an elaborate focus on interesting gameplay, rewarding depth of play and technical quality.<br /><br />Trap Them: http://store.steampowered.com/app/375930
fluffrabbit
Guest
« Reply #19 on: June 10, 2019, 04:27:37 PM »

If it's more about a classic game experience for you, I could see advanced AI distracting the hell out of the player, like how units in Crysis would blow themselves up because they didn't know "fire bad", or the opposite case where the AI is perfect at everything and the game is unplayable. I dunno about all that though; between you and me I'm not much of a gamer.
Logged
Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic