next »

[Blademasterbobo]

this isn't exactly new information. there was another game last year that had some similar diagnostics stuff, and they got similar results for judge playtime and so on.

[phubans]

Wow, that's some bullshit. I'm forwarding this article to my entire team.

[allen]

that is hardly surprising, glad they made a post about it.

people _really_ should put tracking algorithms in their games, and then publicly share the data. that's the only way IGF will do something about this.

[Hangedman]

Ok, I'll share my experience.

Being a flash game, I hosted a secret web version for judges who didn't want to download the swf. It also had the readme on the page, so I meant for it to be easy.
I can only speak to the judges that played the web version. Analytics.

I'd estimate the first third of my game (the compo version was just that part) to be 5-20 minutes long, depending on willingness to read and explore.

Of the 4 judges who played the web version, they played: 5:02, 11:44, 13:39, and 22:23. One came back twice and one came back three times to continue playing, or replay, or etc. Not sure which.

So, not so bad. I didn't expect worse, but I knew it was a possibility.
It really does have to be a demo. Not to apologize for them, 5 minutes should be the barest minimum that any judge plays, but yeah. I didn't expect them to play for an hour, even if my game was that long.

For all I know, the other X judges didn't even look at it. But I'm hopeful they did.

Did I get my money's worth? Couldn't tell you. Haven't gotten anything back.

[alastair]

Doesn't surprise me about Kale in Dinoland, the game looks like it was designed for Gameboy but its on an iphone (that has no buttons lol).

[st33d]

I pretty much thought the same thing. Virtual buttons - kill it with fire.

But still - do your fucking job. What's the point in ponying up so much money if people aren't gonna do their job.

Even if it's shit - play it, godammned dunk your head in that shit and come up screaming, "I fucking hate this game!"

[Glaiel-Gamer]

Even if it's shit - play it, godammned dunk your head in that shit and come up screaming, "I fucking hate this game!"

no

[allen]

what's the point of volunteering to be a judge if you are just going to put in no effort in playing the games? if you don't want to play the games, good or bad, then don't volunteer. simple as that.

goog said this in irc and I don't see any reason why this couldn't be applied to the IGF:

<googoogjoob>   in the interactive fiction competition
 you're obligated to play a game for two hours
 before judging it
<googoogjoob>   and then vote based on your impressions of the first two hours

obviously the time scale would need to be changed, but the principle is the same.

[Glaiel-Gamer]

Ask brandon to be a judge for next year, and play all the games you are assigned to completion

[AndySchatz]

Yknow, I think there's a valid criticsm here, but I always return to the actual finalist results.  Propose which games shouldnt have been nominated and which ones should have.  If the results aren't broken then the process isn't either.  If the results ARE broken, then the IGF should take specific steps to fix those problems.

[allen]

Ask brandon to be a judge for next year, and play all the games you are assigned to completion

Why would anyone volunteer to be a judge if they didn't have the time to play all the games?

If I volunteered to be a judge and I was tasked with completing 20 games, I would complete them. If I knew I wouldn't have time, I wouldn't volunteer. It's not exactly rocket science.

[AndySchatz]

BTW, I actually agree that its a problem that some judges aren't giving any time to their assigned games.  I do think that the past two years of the IGF have suffered from a shift in bias from too much randomness to too much predictability (due to the jury system).  And I think one of the reasons for this problem is that judges aren't giving enough time to unknown games.

HOWEVER, I'm totally willing to defend the idea that some games don't really deserve more than 5-10 minutes.  Also, in some cases, technical requirements make judging difficult, especially in the mobile space.  I did not play one of my assigned games because it required an ipad 2 to play, while I have an ipad 1.

There are way too many games to try, and judges need to be able to focus their attention on the upper eschalon of games.  IMO the best way to improve the discovery process is to improve the social/discussion process of the judging backend.  Applying a bit more pressure on judges to play all of their assigned games will help too.  But I wouldnt go anywhere close to saying that the IGF is broken -- that seems a bit dramatic... I know, it's the internet, everything is dramatic here!

[Dragonmaw]

Given the reduced importance of judges over the past few years, I don't know how to feel about judges not playing games. That being said, i try to play the game until I feel like I want to stop. Some games that means for a long time (or in bastion's case, until the end of the IGF demo; boy, that was abrupt) and some games that means a woefully short period of time.

While you can bemoan that people should always play the game to completion before making a judgement, there's definitely some validity behind playing until you feel like you can't play it any more because it sucks, is boring, you've seen all you need to, etc. Games should reveal at least a portion of their character fairly quickly and be paced well enough to show the player its secrets. Players should not have to wade through hip-deep garbage to find the single, poorly-cut gem in the sewer that is your game.

But since the jury has far more importance than the judges, and they play games to completion (presumably), there's less importance on judges not playing the game to completion.

[bento_smile]

It would be unreasonable to expect judges to play a game through to completion. But it is reasonable imho to expect more than 5 minutes of play.

[alastair]

Having a strict amount of time that someone has to play a game for is dum (although I do find this whole competition pretty dum anyway lol). Some people are more experienced in particular genres and will be able to quickly tell if a game is low quality or not, so the time for someone to make up their mind will vary from person to person.