Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411512 Posts in 69376 Topics- by 58430 Members - Latest Member: Jesse Webb

April 26, 2024, 04:48:43 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Gaming AI: The Art of Being Not Too Smart
Pages: [1] 2
Print
Author Topic: Gaming AI: The Art of Being Not Too Smart  (Read 2534 times)
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« on: December 12, 2016, 03:34:36 PM »

Most games these days have some kind of AI — Artificial Intelligence — for human players to fight against. But just how strong should your game's AI be? Make it too strong and you discourage players; make it too weak and the game will be boringly easy to win.

A popular solution is to put up an easy/medium/hard/insane switch at the start of the game so the players can choose for themselves. But that means burdening the player with a choice they may not be prepared to make. How on Earth am I supposed to know if the "easy" is not too easy for me? Or if the "hard" is even worth trying at my level of skill?

Truth is, people don't want a uniformly smart or uniformly silly opponent. Instead, what they want is an opponent who is engaging. An ideal opponent should match your individual skill: sensing how good you are, it should be playing more or less at your level. Sometimes it would deliberately yield (but never own up, of course!) and sometimes it would challenge you, but overall it would let you progress: not too fast and not too slow — at just the right pace.

In a word, the perfect AI is as smart as needed to keep you interested in the game... for arbitrary values of "you" Smiley

That's the approach we're taking with our new shuffleboard-like game, BATTLE OF THE PUCKS. The goal of the game is to shove one of your pieces so as to drive the enemy's pieces off the board without losing any of yours. And for this kind of game, the AI can be pretty straightforward: it simply runs simulations of a number of randomized shots and chooses the shot that scores best. Already at a few dozens simulations per move, the computer player is virtually unassailable for humans.

However, In BotP, the game's AI spends more of its spare cycles on measuring the optimality of your moves than on calculating its own. Again, the way it does this is by running multiple simulations of the human opponent's every move and its randomized variations:

- If the human didn't score, just how inapt their move was — how far was it from the nearest move that would score? Beginner players' moves tend to be farther off the mark than those by seasoned players.

- If the human did score, how difficult it was — how many of the move's randomizations would have resulted in the same or similar score? If you make a move that scores a ton, but any slight deviation from it would have lost badly, then it might be that you were just lucky — but a more reasonable conclusion is that you play this game very well.

Taking a weighted average of several of your last moves, BotP constantly adjusts its depth of foresight and precision of strikes to match the level you play at. As you master the game, your AI opponent becomes stronger too — never too easy but never too formidable.

It also matters how to weaken a too-strong AI. Perfect unfailing precision is boring, but so is perfect randomness. If you simply add a random fuzz to every AI move, it won't be very engaging: that might feel mechanically stupid, not humanly stupid. Instead, we're giving our game's AI a character of its own that regular players will soon learn and relate to: a variable-but-remembered preference for certain openings, certain directions, certain ways to hit. We humans like entities that we can relate to: entities with their own faces, their own quirks, their own weaknesses that we can learn — and exploit!

Finally, even with the best adaptive AI, it is important to avoid monotony. Having determined the optimal level of smartness, sometimes the program would make a move well above or well below that level. Surprises make the life worth living! Sure, that sudden superstrong move of your opponent — erasing half of your army with a single hit — was a bummer... but, hey, wasn't it beautiful! And soon after that, the AI's amazingly inapt misstrike when you least expected it, just when you were prepared to lose the game... but went on to win... ooh.  

Yes, I want to Play Again!

Wanna try? BATTLE OF THE PUCKS is now available for pre-release testing! Please comment or PM us if interested.  









Logged
AaronB
Level 2
**



View Profile WWW
« Reply #1 on: December 12, 2016, 11:01:28 PM »

Game AI is such a rewarding coding experience.  The hardest part is making sure the player can't 'game' the AI.  Collisions and AI - I'm excited - please count me in as a tester.
Logged

bateleur
Level 10
*****



View Profile
« Reply #2 on: December 13, 2016, 04:48:04 AM »

And for this kind of game, the AI can be pretty straightforward: it simply runs simulations of a number of randomized shots and chooses the shot that scores best.

Of course, that might reasonably be considered to be cheating, assuming the simulation is perfectly accurate.

Again, the way it does this is by running multiple simulations of the human opponent's every move and its randomized variations

Nice approach. Maybe you could do this for the AI too? Then the simulation for each move could have errors introduced into the direction and force to get around the unfairness of perfect simulation.

Taking a weighted average of several of your last moves, BotP constantly adjusts its depth of foresight and precision of strikes to match the level you play at. As you master the game, your AI opponent becomes stronger too — never too easy but never too formidable.

Might be better to only adjust between games. Otherwise the best strategy might be to play badly early on and wait for the perfect opportunity to suddenly improve and win.
Logged

Krux
Level 2
**



View Profile
« Reply #3 on: December 13, 2016, 05:00:15 AM »

Honestly I hate the idea that the game adapts to your skill level, because then becoming good at the game means that you have to trick the AI to think that you are very bad, so that you can beat it much easier. That is a game style that I do not think is enjoyable at all. I want an AI where the reward is the highest when you are the best in the game according to the rules of the game, not some meta AI that tries to learn how good you are. In my opinion the best way to challenge everyone is the tetris level system start easy, but then become hary very fast. Don't try to be smart and try to find "a good match" for the player, it only makes your game worse.
Logged
Dacke
Level 10
*****



View Profile
« Reply #4 on: December 13, 2016, 05:15:59 AM »

If you want to dynamically set the difficulty, without creating weird meta effects, make it explicit. You could suggest a difficulty rating after games or simulate a rating system (like in online chess/go/etc.).

But making the opponent a continuously moving target seems like a terrible idea, because the player won't get any positive feedback when they improve (like winning more, getting a higher rating or being able to defeat a previously too difficult opponent)
Logged

programming • free software
animal liberation • veganism
anarcho-communism • intersectionality • feminism
AaronB
Level 2
**



View Profile WWW
« Reply #5 on: December 13, 2016, 04:08:22 PM »

For a game like this you do need to make a decision on how perfect the AI will be, but as others have stated, making it adaptive isn't a good idea. Better to go with a fixed probability (which will still give surprises) for each level.  That way a player can have a real sense of progression.

BTW this game sounds ideal for online multiplayer.
Logged

gimymblert
Level 10
*****


The archivest master, leader of all documents


View Profile
« Reply #6 on: December 13, 2016, 09:03:50 PM »

Really short way to put it: Ai is level design

Ai is literally procedural placement of difficulty according to progression, each level must be design carefully to hit the flow and learning ramp. What you want is the illusion of dynamism.
Logged

Dacke
Level 10
*****



View Profile
« Reply #7 on: December 14, 2016, 01:46:53 AM »

Really short way to put it: Ai is level design

 Hand Clap Grin
Logged

programming • free software
animal liberation • veganism
anarcho-communism • intersectionality • feminism
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #8 on: December 18, 2016, 12:48:36 PM »

We envision this AI as a kind of a trick to hook casual and beginner players who can immediately enjoy the game without feeling it too hard or too easy, and without burdening them with choices before they are informed and hooked enough to make them. Later, a regular player will be able to adjust their desired level of intelligence in options, and to lock it from any learning drift.
Logged
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #9 on: December 18, 2016, 01:15:14 PM »

Game AI is such a rewarding coding experience.  The hardest part is making sure the player can't 'game' the AI.  Collisions and AI - I'm excited - please count me in as a tester.

Great! PM sent.
Logged
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #10 on: December 18, 2016, 01:33:54 PM »

Nice approach. Maybe you could do this for the AI too? Then the simulation for each move could have errors introduced into the direction and force to get around the unfairness of perfect simulation.

Yes, the AI has some wiggle room too, and sometimes should make mistakes. The lower is the calculated "smartness of the AI", the more mistakes it should make, letting even a weak human win.

Might be better to only adjust between games. Otherwise the best strategy might be to play badly early on and wait for the perfect opportunity to suddenly improve and win.

Not with this type of game, I think. We have many short levels, each round lasts only a few minutes. The first person to shot always has an advantage for that round but once you lose your turn, you might not get it until the end of the round if your opponent plays wisely and doesn't make a mistake. And there are many levels to complete (we currently have 20+). So even if you play badly on purpose during the few easiest levels you will still have a long road ahead, and as soon as you start playing better, the AI will catch up and improve its smartness as well.

Besides, it's not in human nature to play on a shallow level Smiley Why would you keep playing a game at all if not to master it and outperform your own past scores? So if you want to play to your full potential (as most people would do), the game will match your skills accordingly. But if you are sick, tired or just want to kill time without much effort, then you will be able to do so as well.

Also, along with the short-memory tactical AI that only looks at your last few moves and adjusts the current strength of the response, we are adding a long-memory strategic AI that assesses your long-term gameplay and, once you clear a certain threshold, announces loud and clear that you have ascended. That means prizes, fanfares, unlocking more difficult levels (which are too hard for a beginner player anyway), and yes, displaying some proud label/mark. We do believe that positive reinforcement works; negative, not so much.
Logged
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #11 on: December 18, 2016, 04:28:49 PM »

If you want to dynamically set the difficulty, without creating weird meta effects, make it explicit. You could suggest a difficulty rating after games or simulate a rating system (like in online chess/go/etc.).

Yes - this is something we plan to add - but only after you finished the whole game or at least some big enough chunk of it. That is, we don't want to annoy the player with a constant "hey, you're good, so I'm getting smarter, oops, now I'm getting dumber" display, but when all is said a done, letting the player know their average level of skill in this game (in a non-condescending way) will add value.

But making the opponent a continuously moving target seems like a terrible idea, because the player won't get any positive feedback when they improve (like winning more, getting a higher rating or being able to defeat a previously too difficult opponent)

I understand this logic but I don't think it's really a problem with our game. Let's assume the adaptive AI is perfect in the sense that the overall game score is always even (in reality it's not perfect, of course, and there's a lag, so there's always a chance for human to get ahead or to slip back). But even if it's even, the score isn't everything: you're always aware if you won this round because you really did a spectacular hit, or because you played shitty but the computer also did a stupid move in response. In other words, the AI does not alter the physics, and you see at once how well you hit or miss, so you do have the motivation to improve. It's only that this motivation isn't pounded on you by means of the score. And, as I said elsewhere, at the end of the game you do get your reward: there's an average mark of how well you played, and regardless of the score, clearing a certain play quality threshold will unlock more boards and levels to play.

In other words, there are many "naïve" players (think children) who play mostly for the score, so the AI will let them enjoy the game without the score getting ridiculously bad or good. The AI is a way to sustain some healthy drama around the score, without it becoming a tragedy. But the minority of player who get interested and start playing for the art of it, they will get other reinforcements and motivation, and if they want they can always get to the options and turn the adaptive AI off.
Logged
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #12 on: December 18, 2016, 04:36:51 PM »

Ai is literally procedural placement of difficulty according to progression, each level must be design carefully to hit the flow and learning ramp. What you want is the illusion of dynamism.

Yes - except that we don't want the difficulty to be rigidly tied to a specific level (in our game, a specific board/layout). There are easy and difficult boards, but a player should be able to play a wide enough selection of boards regardless of their current skill level, and the adaptive AI is our way to achieve this. There's still incentive to self-improve though: as you genuinely get better, the AI will unlock even more difficult and fancy levels to play.
Logged
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #13 on: December 18, 2016, 04:38:20 PM »

For a game like this you do need to make a decision on how perfect the AI will be, but as others have stated, making it adaptive isn't a good idea. Better to go with a fixed probability (which will still give surprises) for each level.  That way a player can have a real sense of progression.

In a way, this already happens simply by virtue of level design: at the first primitive levels, there's little room for AI to show off its skill - so for the player, the difference between low AI and high AI is less noticeable. In the more difficult levels, though, high AI really shines, finding insanely effective pushes in a crowded field, and you really have to improve just to keep up with it (or rather, since it's adaptive, to keep it up at its best).
Logged
gimymblert
Level 10
*****


The archivest master, leader of all documents


View Profile
« Reply #14 on: December 18, 2016, 06:32:44 PM »

Ai is literally procedural placement of difficulty according to progression, each level must be design carefully to hit the flow and learning ramp. What you want is the illusion of dynamism.

Yes - except that we don't want the difficulty to be rigidly tied to a specific level (in our game, a specific board/layout). There are easy and difficult boards, but a player should be able to play a wide enough selection of boards regardless of their current skill level, and the adaptive AI is our way to achieve this. There's still incentive to self-improve though: as you genuinely get better, the AI will unlock even more difficult and fancy levels to play.

Level design isn't just topography
Logged

JWK5
Level 9
****

A fool with a tool is an artist.


View Profile
« Reply #15 on: December 19, 2016, 07:50:04 AM »

If I am understanding Gimmy right, what he is trying to tell you is that the AI and level design should be synergetic rather than being handled as completely separate things.

To degree, the AI is the level (i.e. it is part of the overall experience of playing the level).
Logged

My Art Tutorials:
 Here

"Today is victory over yourself of yesterday, tomorrow is victory over lesser men." - Miyamoto Musashi
TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #16 on: December 19, 2016, 02:20:10 PM »

the AI and level design should be synergetic rather than being handled as completely separate things.

I would disagree: the very point of AI (the "I" part) is that it's supposed to adapt to different topographies. There are not two but three distinct entities here: player, AI, level - and the level may be as surprising or difficult for the AI as it is for the player.
Logged
ProgramGamer
Administrator
Level 10
******


aka Mireille


View Profile
« Reply #17 on: December 19, 2016, 06:26:31 PM »

Well, if you consider the AI as another agent in the game world, then yeah. Otherwise, designing levels and enemy behavior together is actually a really good idea.
Logged

AaronB
Level 2
**



View Profile WWW
« Reply #18 on: January 13, 2017, 02:27:45 AM »

Playing Battle of the Pucks on a Nexus 7

Title screen looks really good - the high contrast colors makes for a great first impression.

Round 1
So I launch straight into playing the computer and the computer wins first round - OK I'm cool with that.

Round 2
The next level looks a lot easier, just have to repeat the same shot several times and I'll... D'oh muffed it!!! (sigh) computer wins again without a mistake.

Round 3
Now I start laughing - one puck each side - is the computer opponent trying to tell me something? Surely I can win this one. YES!!!! take that computer AI.  OK it's a pretty shallow victory but what the hey.

Round 4
Looks complicated with three rows.  I find a new strategy, hitting two opponent pucks simultaneously gives me more control at removing them from the board - victory is mine... D'oh! missed the shot! and the computer AI wins again without making a mistake Sad

Round 5
AI wins without making a mistake.

Round 6
AI wins without making a mistake.

Round 7
AI wins without making a mistake.

Round 8
There are a lot of pucks on the board for this round and I'm doing a pretty good job of clearing them.  I'm down to the last two opponent pucks without making a mistake, so out of curiosity I decide to knock my puck of the board deliberately.  The computer AI then proceeds to clear all my pucks with it's last remaining piece without making a single mistake.  Hmmm, I'm thinking this AI chap should go for the Olympics.

Rounds 9 - 11
AI wins without making any mistakes.

Round 12
AI MAKES FIRST MISTAKE!!! but I muff it on the first shot...and lose Sad

Round 13
AI wins with 1 mistake.

Round 14
I WIN!!! - with the AI making one mistake.

Round 15
AI wins without making any mistakes doing some pretty brutal shots.

OK at this point I think I can safely say the the AI is way too strong when playing against a beginner - the opponent feels super human - especially when it pulls of several hard shots in a row.

The animation is smooth and the physics feels good, with perhaps one exception, sometimes it seems the puck is on ice**, especially when moving slowly near the edge of the board - it just keeps on moving, the deceleration seems linear and doesn't feel realistic. At the microscopic level the molecules between the puck and the board are heating up from friction, which in turn raises the friction coefficient, resulting in what should be very non linear deceleration. Also as the puck slows down the interplay between static and kinetic friction becomes more apparent.

** Just took a closer look at the board image and realized it may be ice?  I assume traditionally the game is played on a checker board?

The game is a lot of fun and the multiplayer is great, the AI needs work.  Perhaps a more tolerant random probability of failing the shot that is weighted by the difficulty of the shot (distance of shot + proximity of target puck near edge).


« Last Edit: January 13, 2017, 02:53:24 AM by AaronB » Logged

TheLifemakers
Level 0
**

Lifemakers Studio


View Profile WWW
« Reply #19 on: January 13, 2017, 07:24:50 AM »

Many thanks for your test run and such a detailed report! Beer!

Yes, we did decrease the AI smartness since the time we sent your the APK. Of course, it will require more testing to get the right amount of mistakes on the computer side. For example, I tried our latest version yesterday deliberately making lots of silly mistakes and got about 50%/50% score at the end. And if I play at my best, I might win with a score like 27/23 (we will have 50 different boards for the first release).

We can send you a new version over this weekend (lots of stuff was added and changed recently).

We will think more about how to make the physics more realistic. Yes, our boards are "made of" ice so they have less friction than a traditional wooden checker board.
Logged
Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic