I’ve been exploring using OpenAI’s chatbot
ChatGPT as a gamedev tool a little bit and have come up with some mixed results. Sometimes it is very impressive but other times it only appears impressive but is actually incorrect. Most of the time for me it’s somewhere in the middle area of “That could sometimes be useful I guess.”
Code OptimizationThe first thing I tried was asking it to optimize a bit of code that was a small, simple function that I just thought looked ugly when I went quickly looking for an example to give ChatGPT. It’s a function that takes a 2D point and quad parameters, and returns the minimum distance from the point to a point on the edge of the quad, or 0 if the point is already inside:
I was initially extremely impressed! This code looks nicer, and it says it’s more optimized. I was particularly impressed with how it has no definition for my custom TileNavMeshQuad class, nor was it explicitly told that the pos property of the quad corresponds to the top-left corner of the quad, but from my existing code it correctly guessed how everything is structured.
But wait – this is incorrect; this ignores diagonal distances to a corner of the quad for when the point is both beside and above/below the object. Let’s ask it to fix that. In thinking of how to word my request of what I wanted, I decided to just be pretty vague to see it if understood what I meant. Also as an aside I felt kind of weird being polite to a robot? Like I intentionally didn’t want to personify it. So I dropped the please from my request.
Awesome! It understood what I meant, and now includes diagonals. I was honestly kind of caught up in excitement at this point, thinking I was getting some nicer, optimized code. But I notice one more error, it doesn’t return 0 if the point is in the quad like my original function did so I tell it that and it fixes that problem:
Ok perfect! ChatGPT has rewritten my code to do the same thing but faster. So I go to test just how much faster…
And I find that it’s in fact about 5x SLOWER than my original code. So I tell it:
Cool, it adjusts the order of some checks to improve the performance. I do another test and this is faster, but still 1.5x slower than my original code. I didn’t want to just say “faster” again, maybe I should say “as fast as possible?”
Here ChatGPT stored some calculations in some variables once so that they don’t need to be calculated “every time,” but in this case they’re still only used once. So this basically does nothing. I tell it:
Some very general advice that doesn’t help here. It’s at this point I’m realizing none of this really helped me at all. I took another look at my original code and noticed it was in fact already very optimized. I know ChatGPT can optimize ugly code from seeing some examples from other people, but it doesn’t actually test out code. It seems to just confidently provide some code and tells you it’s optimized when the ideal response to my original prompt should be “I don’t know how to make that faster.”
Idea BrainstormingI switch up tasks and think more about things that would actually help me in my current project. Maybe some brainstorming?
This was pretty cool. I had already thought of pretty much all of these things, except for pools of water and cobwebs, which are good ideas I may now consider. The drawback is it’s all fairly generic ideas, so you’ll still need some individual creativity if you want your game to stand out more, but perhaps this could be a handy time-saver for some tasks? The alternative is either googling or browsing various sites with collection of concept art or photos for inspiration.
I try another example:
A pretty extensive list of monsters, but they are of course fairly generic. It does provide some nice little explanations of why they are relevant to a desert.
I decide to do a little test. I decide on a new area for monsters, and first give myself one minute to brainstorm on my own to which compare to ChatGPT’s. I decide as an example a swamp environment, and in one minute came up with:
toads/frogs
witches
mosquitoes
giant bugs
giant birds
snakes
leaping fish
spiders
mud golems
I then ask ChatGPT to do the same:
Not bad. There is a bit of overlap in our ideas, but ideas like slimes, goblins/lizardfolk, the undeads that were trapped, and large predators are all pretty different than what I came up with. Nothing amazing, but for an instant response from just typing a response, it could be a nice springboard for ideas on a new area.
I feel like I’m not using ChatGPT to it’s full potential, which includes longer conversations, so I provide a more unique task and ask it to elaborate more on the swamp monsters:
Some possible fun ideas to consider! I like how it tries to explain the connection between the monsters and ruins, providing reasons for their existence and behaviors. This is something that my quick barebones list was lacking.
Pros and ConsI’m still searching for ways I would actually use this and so far the brainstorming ideas seems to be the most helpful. I think tasks like that may work best since they are very open-ended and there is no one correct answer, unlike most coding problems, where ChatGPT may confidently give you an incorrect answer.
It is pretty helpful at explaining code, however, as well as spotting errors. These could be a nice use case. In one test I took
this random reddit post of someone asking for help with a complex regex expression, copy and pasted their post verbatim to ChatGPT, and was given the same correct solution as the top commenter of that thread, which was impressive. ChatGPT could possibly drastically reduce the number of reddit and stackoverflow threads of “why isn’t this code working?”
There are a handful of cons as well, however. In some other attempts I asked for some help with a general issue I faced with some collisions in Unity, asking how I could achieve something, and ChatGPT explained a non existent feature of “conditional fields” for physics layers.
Sometimes people talk about ChatGPT as if someone without programming knowledge can use it to make a game which is far from the truth. You still need knowledge to test it’s answers, and tell it when it got things wrong. For me I was originally most interested in using it for coding tasks but that seems unlikely now, aside from very occasionally tasks.
It is also still limited in scope to what you give it. I keep an (embarrassingly long) todo list of tasks I need to do to fix things in my game, and went through it trying to find tasks that ChatGPT could help with and didn’t really find anything. I tried a few but didn’t have any luck getting something useful. The main problem is that for example my todo task of “fix game pausing issues during cutscenes” involves my cutscene system, entity movements, and other map updates spread through hundreds of lines of code across dozens of different classes and subclasses that’s needed just to understand the system enough to fix the problem. I’m sure it won’t be long until a bot can be given access to your entire project for this however.
ChatGPT is so open-ended on what it’s capable of that it’s kind of difficult to think of new use-cases, so I’m sure new ones will pop up as people continue to explore its capabilities and as the model is improved. It’s also worth mentioning that ChatGPT itself is a tool that can take some practice to efficiently use. Currently I don’t think it’s a perfect magic answer to any individual problem, although it does show promise with being helpful at some tasks.
Developer
