next »

[ஒழுக்கின்மை (Paul Eres)]

(The design doc tutorial got me on a tutorial kick I guess. I also posted this on my site, but I'll cross-post it here too.)

1. The more playtesters, the better, because you can see what a wider variety of people think about your game. Try not to pick only people who are "gamers", as what's pleasing (or obvious) to them might not be pleasing (or obvious) to others who don't play games as much.

2. It's better to watch someone play your game than to read a report about it. What you can learn from watching people play your game you simply can't learn from what other people tell you via email or instant messenger about your game, and I kind of even feel sorry for games which are released without the developer watching anyone play it in front of them -- usually one can tell when they didn't.

3. Constantly add new eyes. Adding new playtesters with each newer version of the game helps: when someone has played earlier versions of your game they tend to look at it differently from someone who hasn't seen the versions without all the current features, since first impressions develop very differently if the first version they played was 10% done or 90% done.

4. When it comes to taking their suggestions seriously, don't do that. Ignore most of what they recommend -- it works better that way. Use what they say as raw data to make your own decisions about the game, and do think about their suggestions, but keep in mind that game players usually don't know much about game development. They do know what they find fun, what they had trouble with, what was confusing, and what they found annoying. Those are the things you're looking for, not suggestions about new weapons or new gameplay elements, you're still the author of the game, don't make it a design-by-committee game.

5. Spend a lot of time playtesting, preferably constantly; I probably spend at least ten percent of my current game's development time watching people play it or talking about it with people who have played it. Think of it this way: you want people to have fun with your game, so what's the best way to ensure that they will? Let them play it to learn what they find fun and what they don't. It's simple empiricism.

6. Your team, and you yourself, do *not* count as playtesters. You can play your game for fun and look for bugs, but you know far too much about it to be a good judge of many aspects about the game, particularly its difficulty level and how confusing or easy to learn it is. Likewise, what's intuitive to you about the control scheme or even the strategy isn't intuitive to people who didn't make the game. Make sure people can figure it out on their own, just from the game, without your advice.

I found this quote by Shigeru Miyamoto in an interview: "If a fan makes a suggestion, I will often put it in my mind, and I will take in whatever comment I feel is useful. But I make my own predictions of how a user might react to the games I create, and I would say I am sensitive to whether those reactions are in line with what I predicted. People generally have different views and opinions about anything. So I would only listen to whatever information is useful for me. It is interesting to hear what other people say. But instead of reading the blogs, I would rather stand behind a person playing the games and sense how the player is reacting to the game -- whether he is unhappy with the games, or if he is having fun. I can feel all of that directly. It is more useful for me to do that than to read what he thinks of it."

For those of you who want to see a great example of how games should be playtested, read this series of posts by Auriea of Tale-of-Tales, they're wonderful.

[Powergloved Andy]

playtesting is hard work, and sometimes it's hard to choose the right testers, too. It also really depends on the game. Good advice, rinkuhero!

[Don Andy]

4. When it comes to taking their suggestions seriously, don't do that. Ignore most of what they recommend -- it works better that way. Use what they say as raw data to make your own decisions about the game, and do think about their suggestions, but keep in mind that game players usually don't know much about game development. They do know what they find fun, what they had trouble with, what was confusing, and what they found annoying. Those are the things you're looking for, not suggestions about new weapons or new gameplay elements, you're still the author of the game, don't make it a design-by-committee game.

I probably should print this out and hang it in a big golden frame above my monitor.

[KennEH!]

How do you implement watching people play? Seems full of problems. Same with ignoring suggestions and comments. IF it comes up enough isn't that a big sign?

[ஒழுக்கின்மை (Paul Eres)]

I didn't say to ignore "comments", I said to ignore "suggestions" -- there's a big difference. A suggestion is "make an enemy that breaths fire", a comment is "this enemy is hard to see".

I'm not sure what you mean by how does one implement watching people play. You, uh, just watch them play. Read Auriea's posts for a concrete example.

[Laremere]

A post in the Love blog really agrees with this: http://news.quelsolaar.com/#post11

The section titled: "People cant have opinions about things they don't know." correlates to #4 very well, which is what made me think of it in the first place.

[ஒழுக்கின்மை (Paul Eres)]

A post in the Love blog really agrees with this: http://news.quelsolaar.com/#post11

The section titled: "People cant have opinions about things they don't know." correlates to #4 very well, which is what made me think of it in the first place.

Yeah, that was exactly what I was talking about. Particularly this part:

This is why you can never ask your users for solutions, only problems, and the problems can usually be solved by removing the problem all together rather then adding functionality to deal with the mess the problem creates.

[Hajo]

Point (4) is important. Yet a whole lot of people will hate you for it, if they find out their "I know how it must be done" feedback is ignored to a big extend.

So do it this way, but don't talk about you doing it this way.

Edit: Good tutorial, I forgot to say. I also like your design document tutorial, rinkuhero.

[ஒழுக்கின்மை (Paul Eres)]

Haha, it'd be dishonest not to talk about it if you're creating a tutorial about playtesting for other game developers.

And usually when people offer me suggestions or ideas to add to a game, I reply with something like: the reason I make games is because I have ideas for games, and it's strange to assume that I'd need to get ideas from others, and that it's better to make one's own game than to make someone else's game for them.

[Hajo]

I must remember that for future projects

[Wilson Saunders]

Thank you for posting these tutorials rinkuhero (I also read the design Doc). I often do not look beyond the next coding challenge or art requirement. Getting exposed to the good design principles like this is both fun to read and enlightening. Please keep them coming.

[KennEH!]

I didn't say to ignore "comments", I said to ignore "suggestions" -- there's a big difference. A suggestion is "make an enemy that breaths fire", a comment is "this enemy is hard to see".

I see what you mean now.

I'm not sure what you mean by how does one implement watching people play. You, uh, just watch them play. Read Auriea's posts for a concrete example.

Well how many playtesters are you getting? If you have like five, I guees you can watch them play, but only for so long. If you have more you can't have each person send there whole play time, nothing could ever get done. Even more so how can you get too see you like 20+ playtesters unless they are all in your town/city?

[ஒழுக்கின்மை (Paul Eres)]

As I said in the tutorial, it takes a lot of time. I'd estimate around 10% of the time I spend making games is spent watching people play my games, more near the end of a game's development, less near the beginning of its development.

I'd say 20+ is definitely better; Immortal Defense had at least 20 offline playtesters (and probably around 100 online ones). If you don't know that many people willing to play your game while you watch, you need more friends or a larger family!

They don't have to all be from your town/city, they just need to be in the area. Tale-of-Tales's playtesting method (which I linked to in the tutorial) was to ask fans of their previous games to come to their house for the day and play the game. I'm sure not all of them are in their particular city, they were probably within driving distance though.

Another alternative is to pay people to playtest your game; a lot of companies do that, and often people don't ask much, most kids would be happy with $10 in exchange for playing your game for an hour or two. In fact, I'm 30, and even I would be happy with that arrangement, it's mutually beneficial for both parties.

[mewse]

I generally agree with everything you've said, Rinkuhero, but I'd like to stress that you really need fresh eyes at every revision of your game, because feedback from anyone who's seen the game before is 'tainted' by that earlier experience, and so is no longer really valid to be interpreted as a fresh opinion.

My typical approach is to get a single person to play my game each week, typically on some weekday evening;  I'll sit beside them while they play, but won't answer any questions they ask.  Instead, I'll make a note of any questions they do ask, of when they seem to be enjoying themselves, of where they get stuck, of when they seem to be bored, etc.  And I'll let them play until they've played for an hour, or decide that they want to stop playing.

At the end of the playtime, I'll ask them what they liked, and what they didn't like.

If you do one of these per week with a different person each time, you start to get a feeling for what's working and what needs to be improved in your game.  (For example, I know that MMORPG Tycoon desperately needs an in-game tutorial.  Of course, I already knew that.  But now I know it times like eleventy billion.)

[brog]

Well how many playtesters are you getting? If you have like five, I guees you can watch them play, but only for so long. If you have more you can't have each person send there whole play time, nothing could ever get done. Even more so how can you get too see you like 20+ playtesters unless they are all in your town/city?

Watching five playtesters in person is immeasurably more valuable than reading hundreds of written reports.  It's not hard.  If you're hanging out with one of your mates, just pull out your laptop and say "take a look at the new game I've been working on".  And sit them down and watch.  Every time that I have done this my game has gotten better by a ridiculous amount.

I can't stress this enough.  Live playtesting is the only way to find out how people react to your game.  Sending it to someone over the internet and asking for feedback does not work.  Players won't know what they misunderstood, so they won't be able to tell you, but if you're watching them in person it's obvious.  You learn so much from watching the game, watching their face.  If you just get words from them it's a lot of guesswork trying to figure out where they wen't wrong and what needs to be done.  Watch them and it's obvious.

It's a lot of fun too.  Especially when you've done a few playtests already and the game's starting to get into shape.  When the tools you've used to try to get people to understand the game start working, and you're watching them predicting exactly what they're going to do next, because you've brought the model in your head of how a player acts into line with the real thing.

Sometimes really unexpected things come out of playtests, too.  There's a bit of text in my game that players don't read - they skim over it, and if you ask them what it said a few seconds later they'll answer incorrectly.  The text explains precisely how one of the systems in the game works; players ignore it until they realise they need to actually know how it works and then they figure it out for themselves.  But if I remove the text, players have a lot more trouble when they start trying to figure it out for themselves - somehow the vague recollection of what it said puts them in the right direction, but when they do figure it out they think they've achieved it all on their own.  Very strange.

[ஒழுக்கின்மை (Paul Eres)]

It might be better to keep the text there until they figured it out for the first time. Text boxes that you can pass up don't work as well for tutorials as text that remains visible.

[brog]

It might be better to keep the text there until they figured it out for the first time. Text boxes that you can pass up don't work as well for tutorials as text that remains visible.

Thanks for the suggestion, but it would be hard to implement in this case.  It's difficult to measure what's going on in someone's head.  The way this game mechanic works, a player can do exactly the right thing by chance without understanding why it worked.  I've tried to set things up so people will end up taking an experimental scientific approach to figuring out how it works (this is actually one of the goals of my design - to encourage this kind of reasoning).  So I can't check whether they've figured it out by checking if they've performed the right action, instead I present situations that will be increasingly difficult if they're not doing the right thing more often than they would by chance, while keeping the pressure low enough that they feel comfortable doing experiments to confirm their ideas about how the system works.  And playtests have shown that this works if I give them that bit of text at the beginning, but if I don't then they won't realise the kind of effect that their actions have on this system and they'll get stuck.

[agj]

Not directly related, but I still recommend reading a book titled The Design of Everyday Things by Donald Norman. It is, as I have gathered, the book that spawned the 'interaction design' field of expertise. It contains a lot of insight on how to align your designs (in this case, games) to meet its users' needs and expectations, and improve usability. I was reminded because of brog's use of the word 'model'. This book taught me about conceptual models, and how they differ in the designer's head, in the user's head and in the actual design.

[mirosurabu]

I don't doubt that having 'right' (and emphasis is on 'right' to note that it's not just having many playtesters that is valuable but it's WHO makes your playtesting group that determines how valuable it is) offline playtesters is better than having online playtesters. I won't argue about whether 10 offline testers is better than 1000 online because I assume that such arguments are not really based on some real data; I assume they are products of extreme will to show that offline playtesters can help you in a different way online can.

But, my point is that I am not really sure that any kind of offline playtesting will bring more valuable data to game development than online would. One might get five or six playtesters and may assume that any later significant development improvement is due to offline playtesting. No, this may not be true.

Playtesting is useful to learn how people react to different challenges in general (which can be useful with game design) - i.e. how they enjoy playing games, what they find fun, what gets them involved, what's annoying in general or for specific group, and so on. The same knowledge you can gain without actually doing playtesting. (When we do experiments we came to conclusions and learn. But others do not repeat experiments in order to gain same knowledge. They just read about experiment.) So these are basically small experiments one can use to improve game design, but they have to carefully designed and executed. Having a group of same five or six playtesters won't do the thing. So, I'd say this can be useful only if you're very serious about your project and when you are ready to pay for playtesting. Alternative can be - learn from other people's experiences. It can even be more economic for you don't really need to repeat experiments when they already exist.

Further, playtesting is useful for finding bugs and glitches in your game, in general, it's useful for testing whether game runs as one has predicted. This is most common use and it's the main use of online playtesting. The difference between online and offline are minimal, if we're following my raw estimation. Truth to be said, when you don't watch how your game runs when played by someone else in different digital environment it is more probable you will miss something. But when done carefully, online will give same results as online.

So, I'd say one does not really need offline playtesting to develop successful game. Further, I'd also like to say that there are people who wrongly attribute development improvements to random things they did during development. This can apply to offline playtesting.

[ஒழுக்கின்மை (Paul Eres)]

I think that's not the most common use, and if it is, they're using it wrong. Playtesting is not to find out bugs and glitches and to make sure it performs as expected -- that's quality assurance, that's bugtesting, that's not playtesting. Playtesting is when you're testing how a game *plays* (there's a reason it has a separate name from beta testing, it's not just a synonym for beta testing).

The information you can't gain from online playtesting is how players react to your game: what they find fun and what they don't. That information is much harder to acquire if you don't have them in front of you, because people often don't express in words what they found fun and what they didn't, and when and how they were frustrated -- communication through text leaves a lot out, whereas if you watch them, you can see in front of you when they are and aren't having fun, and what they liked and didn't like.