next »

[joshg]

Do any of you guys bother using test-driven development (TDD) in coding your games?  I've done enough reading up on good software engineering practices that I have this nagging feeling it would help me in the long run, but I don't really know if there's a point at which a project is too small for it to be worthwhile.  And unfortunately all the larger projects I've worked on hadn't made the effort to put it in place.

If any of you do go the TDD route, what test setups do you use?  Any recommendations, specifically for a C++ project done in Visual C++ Express '08?

[PaulMorel]

Personally, I think that TDD is just a buzzword.

I think that programming is like any other discipline.  To do the job well, you need to plan thoroughly.  Plan out everything that will be in the final product, every feature, every resource and every option.

Then, just stick to that plan.

In my opinion, how you stick to your plan is far less important than making a thorough plan.  Whether you choose iterative development, test driven development, or any other variation ... as long as you plan well and stick to your plan, you will come out on top.

[dmoonfire]

Speaking from the non-gaming side, TDD is very nice when you have a model (as opposed to monolithic) to work with. On the Linux side, I put my (n|j)unit tests as part of the build process. It only takes a few seconds to run a couple hundred tests and it tells me if I break something. And I frequently write tests for the model before I do the code, since it lets me get a feel for how I'm actually going to use it as opposed to what I think I'll needed.

Another advantage of test-driven is that it encourages you to build functions that do one thing well, because it is easier to test one thing well than tons of everything.

Of course, I use a model/view approach to things, which really helps. And I'm used to working with rather complicated systems. In the game development, at least as far as I got, I found it useful since I try to keep my game model separate from the view. And making sure I didn't accidentally break scoring while fixing a death spiral helped. Or a damn bug where I accidentally did log(0) as part of the score.

For C++, I would probably start with google-fu on "c++ unit test visual studio" and go from there.

[joshg]

Personally, I think that TDD is just a buzzword.

I think that programming is like any other discipline.  To do the job well, you need to plan thoroughly.  Plan out everything that will be in the final product, every feature, every resource and every option.

Then, just stick to that plan.

In my opinion, how you stick to your plan is far less important than making a thorough plan.  Whether you choose iterative development, test driven development, or any other variation ... as long as you plan well and stick to your plan, you will come out on top.

I love how that describes the classic waterfall model of software design and completely doesn't describe good iterative development at all.

[PaulMorel]

I love how that describes the classic waterfall model of software design and completely doesn't describe good iterative development at all.

Are you going to explain/defend that statement, or just put it out there?

I don't even know what 'the classic waterfall model of software design' is.  All I know is what I've learned from developing and deploying about a dozen business apps, a handful of websites, and half-a-dozen hobby apps.  Take it as you will.

[joshg]

I love how that describes the classic waterfall model of software design and completely doesn't describe good iterative development at all.

Are you going to explain/defend that statement, or just put it out there?

I don't even know what 'the classic waterfall model of software design' is.  All I know is what I've learned from developing and deploying about a dozen business apps, a handful of websites, and half-a-dozen hobby apps.  Take it as you will.

Sure.

http://en.wikipedia.org/wiki/Waterfall_model

http://en.wikipedia.org/wiki/Iterative_and_incremental_development

Generally speaking the entire point of iterative development is to go through repeated rapid cycles of development without trying to finalize every feature and detail ahead of time.  ie. To make decisions by prototyping and testing out the solutions rather than trying to work out everything on paper first.  (Not to say that you don't plan at all, but you don't "Plan out everything that will be in the final product, every feature, every resource and every option.")

Just because something is a buzzword doesn't mean it's entirely meaningless.  I don't think TDD, agile programming, or even object-oriented programming are magic bullets that solve all programming problems, but they can all be useful tools if you know when and why to use them.  Sometimes they're even fairly common-sense once you get past the hype.

Man, anyone else remember when OOP was The Buzzword?  I was just barely starting at coding back then ... now people assume that OOP is the Only Way To Program.

[PaulMorel]

I see your point, but I totally disagree.

Generally speaking the entire point of iterative development is to go through repeated rapid cycles of development without trying to finalize every feature and detail ahead of time.  ie. To make decisions by prototyping and testing out the solutions rather than trying to work out everything on paper first.  (Not to say that you don't plan at all, but you don't "Plan out everything that will be in the final product, every feature, every resource and every option.")

In my opinion, iterative development as you describe it, is a recipe for failure.  I speak from experience as a solo developer, a programmer in a team of 25, and a lead developer with a team of 5.

Whenever someone tells me that they don't need to make a detailed plan because they will work it out as they go, it has always been an excuse for laziness.  Past bosses of mine (before I took over development at my current company) have tried to sell me on iterative development and a dozen other development 'strategies', and it has never worked out.  It has been a strategy for pushing off today's work, but trading it for 10 times the work in 6 months.

Iterative planning as you describe it is fine in theory, but in practice, when you commit to spending less time planning, you don't end up with a smaller plan.  You end up with an incomplete plan.

As a result of this incomplete plan, the programmers implement support classes that are only designed to do part of what the final program has to do.  So, when the features that were left off the original plan finally have to be implemented, they have to be hacked into an implementation that can't support them.  Or else the base classes have to be completely rewritten.

And again, I'm just speaking from my practical, real-world experiences.

In my experience, what you end up with when you shorten the planning stage is hacked together software that is hard to support and maintain.  Adding features takes additional time because the programmers are putting hacks upon hacks upon hacks.  Fixing bugs also takes longer for the same reason.

Just because something is a buzzword doesn't mean it's entirely meaningless.  I don't think TDD, agile programming, or even object-oriented programming are magic bullets that solve all programming problems, but they can all be useful tools if you know when and why to use them.  Sometimes they're even fairly common-sense once you get past the hype.

True.

My argument is that none of these development strategies are a replacement for good old fashioned elbow grease.  If you try to use one strategy to save man-hours, or to get around the development tasks that you don't like, then you are shooting yourself in the foot.

Man, anyone else remember when OOP was The Buzzword?  I was just barely starting at coding back then ... now people assume that OOP is the Only Way To Program.

I never even knew what non-OOP programming was until I got out of college and the only job I could get was in COBOL! 

Anyway, I hope I didn't take this thread too far off-topic.  I don't know too much about TDD, and all I wanted to say is that if you use it to abrogate your planning time, then you are making a mistake.

[dmoonfire]

There are a few things with agile (rapid cycles) development where you don't plan out everything, but you still have an end result or goal. You just work in smaller chunks to get something done now, then add on to it. It still requires a fairly good designer/architect to think at least a few iterations ahead and it requires you to break down your tasks fairly well, but that is partially what the work breakdown structure and other methodologies are all about.

I've worked very well in agile environments and I think it works out very well, for a given culture, group, and arrangement. I've also seen it not work in almost as many projects, but I think it has more to do with the skills and knowledge of everyone around more than a shotgun "it doesn't work."

Test-driven is nice because the architect can set up what it should be doing, then you do an iteration setting up the test cases, then the following iterations make them work out.

[joshg]

I see your point, but I totally disagree.
...
In my opinion, iterative development as you describe it, is a recipe for failure. 

I'm not sure you see my point at all.

I never even said that iterative programming is a good thing, or was trying to promote it.  I was saying that your description of "plan everything in detail ahead of time" was incompatible with an iterative development focus - as in they're just two different things. 

You were saying that as long as you plan everything out in full ahead of time, then you can do iterative development as well and it'll work.  Which doesn't really make sense - it's like saying, "Hey, as long as you paint that picture entirely red, then you can paint it as blue as you want and it'll be great."

I, uh, don't even know why I'm bothering with this conversation anymore, so I'll just stop now.

[GlingGling]

I have only a little experience on working on fairly large (4 and 5 months development) solo and team projects. But I understand a lot of what PaulMorel is saying and I think you are misunderstanding him joshg. He is not promoting a particular model.

What he is promoting is thorough well thought out planning. He goes on to say how it is not entirely important to follow your plan. In my mind the objective here is to fully solidify your goal. In my experience developing system this is absolutely the most important step. Once you identify where you are and what you want only then can you start filling in how to get there. A thorough plan on how to get there (using whatever method of development you're comfortable with) puts the project in perspective.

As for your original post, sorry but I can't help you there. But I implore you to be ambitious and try it, after you've gone through the planning process of course. 

[Robotacon]

PaulMore:
I know where you're coming from. I too have been working in big development projects spanning several years with 60+ staff members. For this kind of development you have to model the entire system and have designated business modelers, implementing teams and test teams.

However... If you're creating a game you're not solving for requirements that has anything to do with any business model. Instead you need to put the player experience in the center be that your own will for a personal expression or mainstream success. So you really can't plan everything and stick to that plan.
You need to create several releases and try them out to see if they are as much fun as you initially thought.

I think the discussion we have about the Design Document brings up these topics rather well.

For everybody:
There's a really easy way to prove your point in this discussion. Show us the game you made using the method you promote.

[joshg]

Man, what in the world ... I never disagreed with Paul's assertion that planning is a good thing.  You guys are all crazy.

I poked fun at Paul because his response was to write off TDD as some buzzword that he didn't need to understand, and in doing so he kind of didn't make sense. 

I mean, "plan everything and then stick to that plan" is pretty much by definition non-iterative development.  Note the emphasis.  In iterative development you can still plan ahead, you just review those plans on a regular basis and make improvements.

My argument is that none of these development strategies are a replacement for good old fashioned elbow grease.  If you try to use one strategy to save man-hours, or to get around the development tasks that you don't like, then you are shooting yourself in the foot.
...
I don't know too much about TDD, and all I wanted to say is that if you use it to abrogate your planning time, then you are making a mistake.

For those who have no idea what test-driven development is, let me clarify something: no one in their right mind, ever, would think that TDD is somehow supposed to be a "replacement for elbow grease" by lazy programmers.  The only reason I'm even asking here is because starting up with TDD is friggin' hard work and I'm trying to find out if there are positive stories of it working well for indie-scale games to justify the extra effort.

TDD = EXTRA initial effort.  Just in case that wasn't obvious enough.

(edited to clarify some junk)

[joshg]

Back on topic, I did find out that apparently Visual Studio 2008 has built-in unit testing features, but only in the Professional version and not in Express.

[PaulMorel]

Yes, clearly the answer must be that I have no idea what I'm talking about.  *sigh*  Why even bring up the subject if you're just going to make fun of people who disagree with you?

As I said before, I'm not trying to make some grand argument against TDD, I was only trying to relay my real world experiences with similar buzzworthy development strategies.  I was only trying to relate my experience and opinion that such strategies often distract real-world programmers from the work that needs to be done, and end up causing more work.

If you want to extrapolate that into something to get offended about, then go nuts!

[joshg]

If you want to extrapolate that into something to get offended about, then go nuts!

I'm not really offended, just frustrated that people keep misreading what I'm saying.

Also, I'm agreeing with you for the most part and I should've just kept my snark levels down.

[Zaphos]

Yes, clearly the answer must be that I have no idea what I'm talking about.

I think pretty much everything you've written in this thread is unrelated to test-driven development?  TDD is not the same as iterative development.  It's about incrementally building unit tests along side the code base.  It has nothing to do with how much you've planned out the development process.

For everybody:
There's a really easy way to prove your point in this discussion. Show us the game you made using the method you promote.

Clearly it's possible to develop under any model; the question is rather, 'which model is more optimal?'  Seeing that someone 'successfully' used a model doesn't really answer this question.

Especially since, on a small scale, I think different ways of working help different people.

The only reason I'm even asking here is because starting up with TDD is friggin' hard work and I'm trying to find out if there are positive stories of it working well for indie-scale games to justify the extra effort.

Yeah, I think the main reason I don't do it for games is just motivational -- writing tests is tricky and it just isn't very fun for me.  Even if it made me twice as productive, I'd be half as motivated to work and it wouldn't help.

[bateleur]

Do any of you guys bother using test-driven development (TDD) in coding your games?

No, but not because it isn't good.

The thing about development methodologies is to understand your own particular strengths and weaknesses as a programmer and choose methodologies to suit the way you work.

Do you suffer from feature creep? Then you need a tight initial spec which you should stick to.

Are you bad at foreseeing problems? Then incremental development will be better for you.

Are you - as Zaphos implies he is - prone to losing motivation? Then choose an approach that produces visible results as soon as possible.

TDD is good for people who work fantastically fast but write fairly loose code. It's also some use for people who hate debugging horrible messes (you might think that would be everyone, but actually I rather enjoy it). It's perfect for industrial scenarios where you need high reliability code but your actual coders are mostly straight out of college and have plenty of talent but no discipline.

What happened to your last project? Did it collapse under its own weight as 500000 lines of unmaintainable code? If so, TDD could be for you.

[ravuya]

I found unit tests most useful as a regression test. Test driven design is probably overkill but I can see why people like it.

[ninjascience]

like another poster, I can't speak to TDD in games, but I found it very useful in business apps.  Especially in the beginning phases of the project where I had a lot of business logic to write before I could even start thinking about rendering it in the view.  I was quickly able to start writing tests to see what small parts of my code where doing.  The TDD part, writing the tests FIRST, really got me thinking about how to break down the tasks in the business logic. 

[joshg]

Do any of you guys bother using test-driven development (TDD) in coding your games?

No, but not because it isn't good.

The thing about development methodologies is to understand your own particular strengths and weaknesses as a programmer and choose methodologies to suit the way you work.

Do you suffer from feature creep? Then you need a tight initial spec which you should stick to.

Are you bad at foreseeing problems? Then incremental development will be better for you.

Are you - as Zaphos implies he is - prone to losing motivation? Then choose an approach that produces visible results as soon as possible.

TDD is good for people who work fantastically fast but write fairly loose code. It's also some use for people who hate debugging horrible messes (you might think that would be everyone, but actually I rather enjoy it). It's perfect for industrial scenarios where you need high reliability code but your actual coders are mostly straight out of college and have plenty of talent but no discipline.

What happened to your last project? Did it collapse under its own weight as 500000 lines of unmaintainable code? If so, TDD could be for you.

This is a great way to look at it, thanks.  My last projects haven't collapsed under their own weight, so probably TDD isn't a huge win for me.

I'm not so much a feature creep person so much as needing to see quicker results, and trying to get over a tendency to over-engineer up front.  At least as far as small game development goes - the over-engineering would be perfectly appropriate for a large business project, probably, but it means small games can take me far too long to get going, which makes it harder to stay motivated.