DevLog Update #60 - 06/03/2014Indy PopCon Post-MortemI exhibited Relativity at
Indy PopCon this past weekend, and decided to write up a bit about my experience.
Indy PopCon is a pop culture and comic convention that took place in Indianapolis, at the Indiana Convention Center from Friday, May 30th to Sunday, June 1st. This was its first year, and I initially heard about the conventions from the guys at The Amiable. At the time, I hadn’t been to a convention before (this was before I even got into the Indie MEGABOOTH at PAX East), and didn’t really know what to expect.
However, I had heard from other developers that conventions are really useful for a number of reasons, and since this one wasn’t very far away, and was relatively inexpensive, I decided to go for it.
By the time Indy PopCon rolled around, I had already shown the game at IndieCade East’s Show & Tell, and also at PAX East, so I had a much better idea of what I wanted to get out of the experience.
Primarily, I wanted to test out the new design changes I had implemented after PAX East, to see if they are effective, and if I’ve resolved the issues I had witnessed then. Since IndieCade East back in February, I’ve been on an intense cycle of playtesting & iterating. Every month, I’ve attended at least one public convention, and each time, it has allowed me to pinpoint different problems with puzzle progression and pacing.
At PAX East, the introductory section of the game was quite good, but I did notice about 4, 5 areas that caused hiccups, and resulted in players getting stuck for longer than necessary. Since then, I had introduced recursive world space, rearranged some of the puzzles, and also tweaked the architecture in the puzzles.
I wanted to use Indy PopCon as an opportunity to see if these changes fixed the problem. With this in mind, and the fact that the game is still a bit ways away from release, I wanted to keep the booth very low-key. I wanted to see how people who hadn’t heard of the game before, and no idea what it was about, how they would react.
Here’s a picture of my setup:
Next to me on one side was Card Dungeon, and on the other side was an empty table. Here’s what it looked like from the other side. I was directly across from Young Horses (
Octodad),
Ragtag Studio (
Ray’s the Dead), and
Lunar Giant (No Kings):
The ConventionIndy PopCon was very different from PAX East. For one thing, it was magnitudes smaller. In large part, this was because it was its first year in operation. Another big difference is that it wasn’t a game-specific convention. There was only a small section made up of game developers (around 25 booths/tables). The majority of the booths from what I could see were related to comics, toys, cosplayers, and other internet stuff. This meant that you had a lot of people who attended to see comics or other things, and who had no interest in games.
This was very noticeable, especially on Friday morning (the crowd on Friday is very different than the ones on the weekend). At PAX, out of the many people I asked to try out the game, I can only recall one person who turned me down. Almost everyone there is interested in games and willing to at least give something a try. At Indy PopCon, I think the first 6 or 7 people I asked just said no right away. It was bit of a rough start.
Things got much better later that day, as well as throughout the weekend, but I think it’s indicative of the attendee demographic.
AnalyticsGenerally, I don’t pay too much attention to analytics, as I don’t think they’re an accurate way to assess the design of a game. I won’t go into the specifics here, but will just link to
this GDC talk from Alex Bruce, in which he talks about some of the problems of letting analytics guide your design.
However, one number I’ve found to be very helpful, especially in convention settings, is seeing how long people play the game for before they stop. The reason why this was important to me, was I wanted to gauge how long the game can keep people engaged in an environment that is quite distracting, with lots of things competing for their attention.
Generally, my line of thinking is, people will stop playing the game once they get bored or frustrated, so if they keep going, it’s a good indication that they are in “flow”. Now, this isn’t perfect by any means, and there are several reasons why someone would quit playing earlier:
1) They simply want to see other things at the convention
2) They get to a point in the game where they realize a much larger time commitment is required.
3) Their friends are waiting for them and want to move on.
In any case, I decided to keep track of everyone who played the game, and how long they played for. Here are the results:
Day 1 - Number of players: 22, Average Playtime: 9 minutesYou can see it was off to a slow start. Several people tried it for a few minutes, and then got up to leave. I think this was largely due to the fact that the convention had just started, so a lot of people are just trying to get an overview of what’s around. I also noticed that many of these people tended to be much older.
As the day went on, I started to have more people sit down and play the game for longer. They also tended more to be “gamer-types”, and were already interested in the premise of a first-person puzzle game.
Day 2 – Number of Players: 33, Average Playtime: 16.3 minutesOn Saturday, the crowd at the convention was significantly larger. There also seemed to be many more gamers on that day. You can see that both the number of players and their playtimes increased. One thing I noticed is that if people got past the first 2 puzzles, they would then keep playing, usually until they got to the exterior of the level, which is the first milestone.
There were a few people who only played for a minute or so. For me, this doesn’t really factor into my consideration for design – most of them were just interested in trying things out very briefly, or just deemed that this wasn’t their kind of game (in the sense that some people only like FPS and others only like RTS). For me, the takeaway here is that once player get past the first few introductory puzzles, then they’re engaged. This is great sign for me, and indicative of the improvements I’ve made with regards to pacing.
Day 3 – Number of players: 17, Average Playtime: 23.1 minutesOn Sunday, the crowd was smaller than the Saturday crowd, but still larger than the Friday crowd. The demographic was also very different, and people felt much more relaxed. My guess is that many of the attendees had already come on either Friday or Saturday as well, and so had already seen man of the exhibits and were therefore not in as much of a rush.
You can see that despite the smaller number of players, the play time was longer. There was a 13-year-old boy who sat down without knowing anything about the game, and played straight through for 85 minutes. He actually got to the very end of the build, in which there was a level that I had just implemented. Since I hadn’t have anyone play that level before, it was ridiculously difficult and tedious. I just told him that he had pretty much beaten the game as it was, and there was no need to continue.
To see that he got so far was really encouraging, and also allowed me to get some feedback on some of the later levels. There were also a few people that brought friends back to try out the game, which was awesome.
Here are the numbers, for all three days:
Total Number of Players: 72
Average Playtime: 14.4 minutesI didn’t keep track of playtimes during PAX East, but I think the average playtime there was closer to 10 minutes.
ConclusionAll in all, Indy PopCon was a lot of fun and a very positive experience. I got to hang out with a bunch of cool developers, met lots of fans, and got plenty of feedback on the game. I did notice a few areas which could be tweaked, but the parts in which players were getting stuck and giving up were fewer than at PAX East.
I think it’s definitely a good sign that the average play time has increased. Again, I want to emphasize that one should not read too much into these numbers. It’s important to understand context, who the players are, what kind of games they play, etc. It’s also important to not rely too much on metrics in design.
In addition to recording the start and end times of the players, I also observed the players playing, and followed up with questions when they were done, asking for suggestions and feedback. This part was tremendously helpful as well.
LogisticsJust a quick note on logistics/prices of exhibiting at Indy PopCon, for those who are interested. The table was $150, and electricity was $80 ($10/amp, and I had two laptops estimated to draw 4 amps each).
Also, special thanks to David Laskey of The Amiable for lending me a laptop for the duration of the convention.