next »

[jeb]

So I'm kind of sad Mon&Bot didn't make it, but I'm really excited by Super Meat Boy, Monaco and as everyone else, Limbo. And all the rest as well 

Congrats and good lucks to all the finalists on TIGS!

Mon & Bot was actually quite similar to Limbo in how it felt and how it played. The huge difference was in puzzle/level design. Limbo was a blast, while Mon & Bot was not. (Neither of the games were on my judging list, so I didn't score them.)

I'm surprised that Mon & Bot didn't end up as a finalist in visual art, because it was extremely atmospheric. However, like I said, the game was tedious and uninteresting, and suffered from bad level design.

I think the reason why Limbo didn't reach the grand prize category was because the version they submitted to IGF was pretty short. But that's just speculation.

[Alex May]

I agree with what Bennett said.  And with what Farbs said about diluting the award with too many finalists.

I'd just add a few more stages to the whole thing.

1. Technical disqualifications - not indie, broken, buggy, tech demo, unplayable
2. Weeder round - obviously bad/mediocre/uninteresting games, try to get it down to < 100 games
3. Democratic vote - pick 10 games for each category
4. Elite panel - pick 5 finalists for each category
5. Final vote - all judges pick their winner for each category

Also, I'd post the process on the results page.  Why not?  It would save Simon/Matthew having to repost it over and over again in each discussion...

Cool ideas... sounds like it would take a bit more admin though.

I'll say that as a judge I enjoyed playing the terrible games too as I felt I could really give some feedback and help the developers improve their game.

The 'bad' games (or games that would generally be considered to be bad) often have more charm and innovation than the 'good' games, so it would be a shame to miss out on those.

[PGGB]

I hope Monaco wins at least one prize. To me it looks like the most interesting of the bunch.

[Jay Jeon]

I'm surprised that Mon & Bot didn't end up as a finalist in visual art, because it was extremely atmospheric. However, like I said, the game was tedious and uninteresting, and suffered from bad level design.

Guillaume,jeb, Thank you for metioning mon & bot.
I know well why mon & bot didn't make it. It was a natural result sadly.
I sould have had more time to design levels.

jeb, Did I have any hope?

[jeb]

Guillaume,jeb, Thank you for metioning mon & bot.
I know well why mon & bot didn't make it. It was a natural result sadly.
I sould have had more time to design levels.

jeb, Did I have any hope?

Hehe, I was maybe too harsh, I was just speculating in why Mon & Bot didn't reach the finals... One thing you could think of is not to repeat the same puzzle too many times. Once the player has figured out how to raise/lower a bridge, it isn't very much fun to do it again.

Also, at some locations, there were misleading dead-end paths. For example, at one place there were some platforms you had to jump across. If you fell down, you would come to a path that went back to the start so you could try again. The problem was that at the start, both the upper and lower paths looked like reasonable options, so if you choose the lower path first you would have to go back again.

I'm really off-topic now... but to summarize: remove repetition and decrease travel time.

EDIT: And to answer your question, sure there was hope! Especially in the visual art category. I doubt you were far off from the top 5.

[Jay Jeon]

you said is right.
actually, I didn't get any feedback from judges yet.
but, finally, I got a great and accurate feedback now.

now I'm gonna make it better.
and thank you jeb again.
I will keep in mind what you said.

[ஒழுக்கின்மை (Paul Eres)]

Cool ideas... sounds like it would take a bit more admin though.

I'll say that as a judge I enjoyed playing the terrible games too as I felt I could really give some feedback and help the developers improve their game.

The 'bad' games (or games that would generally be considered to be bad) often have more charm and innovation than the 'good' games, so it would be a shame to miss out on those.

i also don't think a weeder round would be that good of an idea, for the reasons you mentioned, but i like the disqualifications round, especially if it comes with something that rejects "pseudo-indie" games (games with more than $500k budget, games that already have publishers, and most importantly, games done by people who are being paid a full time salary to do them).

[AndySchatz]

Cool ideas... sounds like it would take a bit more admin though.

I'll say that as a judge I enjoyed playing the terrible games too as I felt I could really give some feedback and help the developers improve their game.

The 'bad' games (or games that would generally be considered to be bad) often have more charm and innovation than the 'good' games, so it would be a shame to miss out on those.

i also don't think a weeder round would be that good of an idea, for the reasons you mentioned, but i like the disqualifications round, especially if it comes with something that rejects "pseudo-indie" games (games with more than $500k budget, games that already have publishers, and most importantly, games done by people who are being paid a full time salary to do them).

There already is this, IIRC.  Last year Matt and Steve played through all the entries first before sending them out to the judges to make sure they ran correctly.  The IGF's eligibility code is totally honor system (which has been the subject of MUCH debate in the past), so in the weeder round they are only looking for games that don't run, they aren't trying to weed out games that aren't "indie".

[c-foo peng]

The way I see it, the main problem with the current system is that there's no real point for comparison.

So say judge A gets 40 random games to judge. His entire point for comparison of games then are those 40 games, and he grades them accordingly to the differing quality between each one. Likely, the best game of the set will get a 10, and the worst will get a 3 or something, it's just how the human mind likes to organize things. Now judge B gets an entirely different set of games, and grades those accordingly. Even though they both have a completely different context for scoring, those scores are going to be counted equally.

The obvious way around this would be multiple rounds, each round ensuring more and more that the judges are judging the same games. So if we reduce the number of games to 10 per category in the first round, then in the second round there's another maximum of 40 games that everyone deliberates on (there will likely be less due to duplicates within category).

This would double the time needed for judging, but I think the results would be much more accurate. Because everyone is playing the same 40 games, and those games are organized into award categories, there can be more detailed discussion amongst judges as well.

[Matthew]

I just fired off the feedback emails!  Looks like we hit 1,000+ pages of feedback (1,430,802 characters).  At a glance it looks like most entries should get ~3 printed pages worth of feedback.  Some judges write more than others, so expect some variation.

Feel free to paste feedback publicly, too, or even use it in marketing (credit to "IGF 2010 Judge")...

[cactus]

I got the judges' feedback for my game just now. I'm pretty impressed. I actually got seven comments ranging from three sentences to almost full reviews (last two years I've had one or two comments). In total there were over 850 words in there

I'm glad I got such positive comments, but I guess you had to do a lot to be among the top scorers in any of the score based categories this year.

The only negative feedback I got was that I didn't include any means of level selection (although I did include a way to skip levels but forgot to mention it anywhere ). Probably wouldn't have made any difference in the end, but still.

Might be just because I finally entered a decent game that I got so many comments, but if that's not the case then the judging process does seem to have improved a lot.

[mirosurabu]

What kind of rating category "Overall" is?

It says "Your game best scored in: Overall". How can something have a best score in overall?

Unless I'm confusing overall with average.

[AndySchatz]

My feedback was really informative too.  Interestingly all of the negative crits were different from one another.  I also got a lot more in-depth feedback than previous years with 1039 words across 10 written comments.

[Glaiel-Gamer]

I got 5 comments, then the rest of the email was identical 4 times copypaste of the first 5 comments. weird

[Alexander Bruce]

I got 3 comments.

One hated the game entirely and didn't really tell me anything more specific. Sounded like they'd played the game for about 10 minutes and then rage quit. Fair enough, but that's not enough information for me to know what to really improve with the game.
 
One praised the game highly, but didn't really tell me where things could be improved. Maybe they didn't think a whole lot needed to be improved, which is fine as I've heard that feedback from others as well, but I know there's still a plethora of things that can make the game better.

One piece of feedback was quite long and incredibly useful, breaking everything down and assessing everything in detail, really showing me what they thought worked and didn't work and why.

[Matthew]

What kind of rating category "Overall" is?

It says "Your game best scored in: Overall". How can something have a best score in overall?

Unless I'm confusing overall with average.

This is what we tell judges:

Treat each of these ratings as completely independent of one another. We have a separate 'Overall' score because sometimes, the whole is more (or less!) than the sum of its parts. If you loved the experience but hated everything in isolation, we want you to be able to give the game a suitable overall score.

[Stegersaurus]

I won't go into the specific feedback I got, but I got a variety of feedback and it's good to have (makes me regret not delaying the release of the game til this feedback, since it seems to have approached a lot of my weaknesses). Most people were very positive about the game, while there were also a few with very specific and informative critiques (sad they didn't like the game, but not wholly surprising anymore). I think the most telling part is that my lowest scored category was "Audio". At this point I really regret not working with an audio engineer on my music-driven game... whoops!

[VinceTwelve]

"I'm going to give you a fairly high score (compared to the other games in my list)"

That's going on the box!

[Riley Adams]

Yay, my feedback wasn't horribly negative! I got 4 comments, they all said generally what I had been expecting: a well fairly implemented mechanic, and somewhat reminiscent of Portal but a bit buggy and rather lacking in personality and audio, also a bit motion sickness inducing .

Couldn't play this one as much as I'd have liked because it really, really started giving me motion sickness trouble.

I scored highest in design and lowest in audio (no surprise with audio, there were like 8 sound effects in the whole thing... Overall, I'm happy I managed to finish the thing in time for the deadline, and between the judges and the public feedback I've gotten I've got a pretty good idea of what I need to work on.

Definitely going to keep going with this project, but I after I get the engine rewritten I think I'm going back to the drawing board with the art style (1 man team + realistic art style = lots of wasted time on shaders for what's ultimately a bland look anyways), and some of the gameplay elements. I also need to find some way to add some personality...

[SephiRok]

Might be just because I finally entered a decent game that I got so many comments, but if that's not the case then the judging process does seem to have improved a lot.

I think that's just because feedback was obligatory this year.

It certainly is nice to finally see the comments though. This was our first year entering, but I cannot imagine it without the feedback; it's a must.

Having said that, we're pretty disappointed in the judging process. As a competitive multiplayer-focused game we got basically ~30 minutes of multiplayer gameplay (from server logs), which can be seen from the type of feedback as well. It seems that if you have an unreleased multiplayer game, and as such with no active community, you have pretty much no chance, as 4/5 judges won't even arrange to have proper match. I guess we just expected too much for $90. No matter how much money the judges get I would expect some more professionalism and respect from all the judges.

So, what we got out of this experience is that you can't really expect anything from judges. Treat them as if they were a clueless customer (at least one didn't even figure out how to add an AI -- I'm not even sure he read the instructions for judges as he said there was no "computer player i could take on"). Admittedly that was also our mistake, we should have made certain things clearer, though at the end most didn't play the game properly anyway. I hope this serves as a warning to others entering in the future as well.

Congratulations again to everyone who got nominated, and good luck.