next »

[stevesan]

measuring very simple statistics like that can be done with steam achievements, indirectly. some games have an achievement for doing different things -- beating the game, beating a particular level, finding a particular thing, or whatever. and steam shows the % of all people who own the game who accomplished that achievement. since most games have a 'beat the game' achievement, you can get a fair sense of the game's difficulty (or length, or boringness, among other factors) by looking at what % of the people who bought the game beat the game

as an example, see this forum's own zaratustra's game eversion; its achievements on steam are interesting: http://steamcommunity.com/stats/Eversion/achievements

92% of people beat level 1
86% beat level 2
84.8% beat level 3
64.6% beat level 4
63.7% beat level 5
61.9% beat level 6
54.6% beat the game in the normal way
29.9% got the best ending

just from looking at that, it's obvious that level 4 is the hardest level, with a drop-off of 20.2% of players who beat level 3 but not level 4. this is followed by level 1 (8% drop-off) and the final level (7.3% drop-off). for the first level the effect may have to do with some people who own the game not even bothering to play it, however, so that can probably be discounted

Oh wow. That explains why so many games give me dumb achievements for completing a level...probably also just designer laziness (I've been guilty of this myself), but that is a very good use for achievements. But then again, on some platforms (iOS) you could just have it send you data every time someone beats a level.

[rek]

as an example, see this forum's own zaratustra's game eversion; its achievements on steam are interesting: http://steamcommunity.com/stats/Eversion/achievements

92% of people beat level 1
86% beat level 2
84.8% beat level 3
64.6% beat level 4
63.7% beat level 5
61.9% beat level 6
54.6% beat the game in the normal way
29.9% got the best ending

just from looking at that, it's obvious that level 4 is the hardest level, with a drop-off of 20.2% of players who beat level 3 but not level 4. this is followed by level 1 (8% drop-off) and the final level (7.3% drop-off). for the first level the effect may have to do with some people who own the game not even bothering to play it, however, so that can probably be discounted

I didn't know this about the achievements paradigm. But I think you need more than one dimension to form this sort of conclusion. If you don't factor in deaths, the drop-off (at any point) can be explained by people getting bored and not continuing – a low death rate followed by a sharp drop-off would also show you that – which is just as likely when the drop-off occurs at the end of level 1.

Oh wow. That explains why so many games give me dumb achievements for completing a level...probably also just designer laziness (I've been guilty of this myself), but that is a very good use for achievements. But then again, on some platforms (iOS) you could just have it send you data every time someone beats a level.

I would hope once the game leaves beta testing this sort of thing would be unnecessary.

[stevesan]

as an example, see this forum's own zaratustra's game eversion; its achievements on steam are interesting: http://steamcommunity.com/stats/Eversion/achievements

92% of people beat level 1
86% beat level 2
84.8% beat level 3
64.6% beat level 4
63.7% beat level 5
61.9% beat level 6
54.6% beat the game in the normal way
29.9% got the best ending

just from looking at that, it's obvious that level 4 is the hardest level, with a drop-off of 20.2% of players who beat level 3 but not level 4. this is followed by level 1 (8% drop-off) and the final level (7.3% drop-off). for the first level the effect may have to do with some people who own the game not even bothering to play it, however, so that can probably be discounted

I didn't know this about the achievements paradigm. But I think you need more than one dimension to form this sort of conclusion. If you don't factor in deaths, the drop-off (at any point) can be explained by people getting bored and not continuing – a low death rate followed by a sharp drop-off would also show you that – which is just as likely when the drop-off occurs at the end of level 1.

Oh wow. That explains why so many games give me dumb achievements for completing a level...probably also just designer laziness (I've been guilty of this myself), but that is a very good use for achievements. But then again, on some platforms (iOS) you could just have it send you data every time someone beats a level.

I would hope once the game leaves beta testing this sort of thing would be unnecessary.

If beta testing was sufficient for everything, Blizzard/Valve wouldn't be constantly tweaking their games after release You should definitely beta test, but you can't catch everything until it's released in the world.