next »

[Christian Knudsen]

And if Phish is telling the truth, that bug only happens to less than 1% of people, which would make it pretty hard to catch.

It bothers me a bit that everyone seems to be down playing "a mere 1%".
I mean imagine if 1% of cars had a bug where the brake would completely fail. People would be dying left and right.
Sure sure, flawed analogy, but still it should be quite clear that even 1% is a huge number of people.

Nobody's downplaying it. It's just that a bug that happens to less than 1% of people is hard for even Microsoft to catch. Which is exactly what the post you quoted said.

[crowe]

My Fez save is gone, for one.


Terrible game, terrible purchase. So disappointing.

[Chris Pavia]

And if Phish is telling the truth, that bug only happens to less than 1% of people, which would make it pretty hard to catch.

It bothers me a bit that everyone seems to be down playing "a mere 1%".
I mean imagine if 1% of cars had a bug where the brake would completely fail. People would be dying left and right.
Sure sure, flawed analogy, but still it should be quite clear that even 1% is a huge number of people.

My statement had nothing at all to do with if the bug was important or not. I stated that a bug that only happens to 1% of people is extremely difficult to catch by testers.

[cskau]

Nobody's downplaying it. It's just that a bug that happens to less than 1% of people is hard for even Microsoft to catch. Which is exactly what the post you quoted said.

It seemed to me that at least Phil was trying to do just that in the talk.

My statement had nothing at all to do with if the bug was important or not. I stated that a bug that only happens to 1% of people is extremely difficult to catch by testers.

I apologize if I was putting words in your mouth there. I guess I was reading too much into your "just 1%".
I completely agree that debugging a case like that can be near impossible as a tester. All I wanted to point out was that to me at least something that "only happens to 1% percent of people" is still rather significant if you're on the receiving end of it as a user.

</devil's advocate>

 

[Zaphos]

That's not the sort of bug cert testing will catch,

If it doesn't even try to catch a huge category of bug (including a very blatant bug, at that), then the bug cert approach seems just blatantly wrong to me... as a consumer, the rapid patches of Steam seems like a way better strategy.

[PsySal]

and yet they missed a bug that deleted entire save files

That's not the sort of bug cert testing will catch, that should have been caught by standard testing by the developer or by the publisher's QA team if there was one.

In other words, $40,000 doesn't buy a heckuvalot.

I still maintain this is a fake fee, in that it doesn't purchase services anywhere near to proportional to the cost of providing those services, yet still claims to be covering the cost of those services. $40,000 should buy you one person-year of testing, which is more testing than a patch realistically would ever need, even in the fantasy world of games that couldn't be easily and automatically patched. We are not talking about cartridges here.

At best it's a deterrence or penalty fee.

[The Monster King]

probably deterrence, to avoid those games that come out and immediately need a thousand patches because they are huge buggy messes (it still doesnt work, im not talking about Fez either there are super buggy things in the world)

sounds too bad for their save breaking bug

[SirNiko]

Thank you XBox players for debugging this for the PC players who will play this in the future.

[Paul Eres]

interestingly it doesn't cost you 40k to patch an xblig game, only an xbla game

[Gabriel Verdon]

It's not really interesting - XBLIG is peer reviewed, not rated by the ESRB, and doesn't require cert.

[Manuel Magalhães]

The 40K patching limitation is arbitrary as heck, even moreso than the WiiWare size limit.

[ham and brie]

It's probably priced at the point where they think that large publishers would choose to do more thorough testing rather than taking a risk that they'll need to issue another patch.

The platform certification testing is not a substitute for the QA the developer and publisher ought to do. The platform holder does want to avoid a perception from users of a lack of quality on their platform, but they're not trying to enforce a perfect record. If a particular game has serious bugs, so long as it doesn't seem to the norm for the platform, it will be the developer's reputation that takes the hit.

[Paul Eres]

It's not really interesting - XBLIG is peer reviewed, not rated by the ESRB, and doesn't require cert.

yeah but like MM mentioned wiiware and psn games don't have anywhere near that kind of fee -- it seems excessive even to charge AAA companies that much to patch a game, let alone indies. also, to my knowledge, games that are patched to not need to be re-rated by the ESRB

[brettchalupa]

Yeah, if there is no change in the content I can't imagine a re-rating being necessary.

[Zaphos]

The platform holder does want to avoid a perception from users of a lack of quality on their platform, but they're not trying to enforce a perfect record. If a particular game has serious bugs, so long as it doesn't seem to the norm for the platform, it will be the developer's reputation that takes the hit.

Does cert actually improve users perception of quality though?  I don't see people generally preferring XBLA versions over Steam/PSN/etc versions as 'less buggy'.

In practice, it just doesn't seem like much of a factor at all, except for the occasional story about how an XBLA version won't get updates/free-stuff because of the fee.  In theory, cert could be making XBLA games seem more buggy: Bugs that do make it through cert will be on XBLA for 2+ months, while bugs on Steam games can be patched in a day -- so way fewer users will see the bugs on the Steam game.