Devlog Update #349 - 11/29/2018Like always, last few months have been really busy. Lots of new people have joined the team as we push for the finish line.
Build Automation Process
One of the problems that have come up near this late stage of development is increasing build times. Early on in development, it was pretty easy and fast to make a build as there just wasn't that much content. Now, however, the entire process takes over an hour.
First, we have to create optimized versions of every scene in the game. This is a process that is specific for Manifold Garden. The scenes of the game are rather large, and the version that I edit with contain a lot of scripts and content that are not needed at runtime. So before a scene is used in the game, we create an optimized version of it, with meshes combined, hierarchy reorganized, etc. This process takes at least 15 seconds for each scene, with some taking as long as a few minutes. In total, it usually takes about 40 minutes to regenerate all the optimized scenes. During this process, Unity is unusable.
Once the optimized scenes are generated, we can then make a build. There are roughly 150 scenes in the current game, and on average it takes about 10 minutes to make a build. Like the process of regenerating the scenes, Unity is unusable during this time.
When the build is finally created, I then zip the folder and upload it to itch.io for the playtesters using the itch tool Butler.
So we have these for steps:
1. generate optimized scenes (40 minutes)
2. make a build (10 minutes)
3. zip (5 minutes)
4. upload (5 - 10 minutes, though it's not necessary I stick around)
We're at the stage now where we're uploading builds pretty much daily, with new fixes and updates for the playtesters. The problem with the above sequence is that I have to keep returning to the computer between each step.
We needed some sort of automated build process to streamline this. Luckily for us, Unity gives us a really great feature, it supports
Command-Line Arguments.
Using these arguments, we can launch unity using “-executeMethod <Method>” which will run one method for us. Using that in combination with “-batchMode” (this runs unity without any windows opening) lets us execute the necessary steps. Another set of arguments to keep in mind is the license-related commands. If you’re deploying to multiple platforms, you can switch between licenses through just arguments, and re-build after switching.
So, using an .NET 4.5 application, We run unity with a bunch of arguments, which then runs the scene optimization code, then builds the game using unity’s
BuildPipeline.BuildPlayer. Then we use .NET’s built-in ZIP management to make a zip file, then run a butler command to upload it to itch.io. We've shifted to doing all the building on a separate machine so we can still work on the game, but the nice thing is we can just set type "build" into a command line and have the rest taken care of.
Debug panelDebug tools are more and more important as we try to pinpoint and fix up all the bugs. Initially, we'd just have a lot of toggles hooked up to random hotkeys - press K for screenshot, press ] for noclip etc. However, this got really hard to manage after a while. While it's still useful to have hotkeys, it's also not practical to have to memorize what 20 of them do. Additionally, keyboard and mouse isn't always available, only a controller.
We now have a series of debug panels that can be used to load specific levels, toggle on and off different states, as well as specific functions to help us diagnose bugs. We can also show the hierarchy and unity log in the game at runtime.
FMOD ProblemsWe're using FMOD for the audio in Manifold Garden. While it's a really powerful tool, especially for the interface that it provides for the sound designer, I'm not sure it was the right choice for us.
I mostly chose to work with FMOD because when I was asking around about audio a few years ago, FMOD and Wwise were two names that came up very often. Both were highly recommended, and I think I ended up going with FMOD for their pricing model (basically free for an indie title).
Initially, it was fairly straightforward to get FMOD integrated into the project and to have a sound in the game. However, no one on the team had ever worked with FMOD before. And as the game grew, and the scope of the FMOD project grew, that's when problems started to come up.
For example, there's the concept of voices in FMOD. The idea is that if you have 1000 objects emitting sounds in the game, you don't necessarily need to play all 1000. You could get the same feeling with much fewer sounds. So while you might have 1000 sound events, only 64 are active. Sound events can be set to "steal", so if you have a new sound start to play, it can use one of the voices from another sound, perhaps the quietest one, or the oldest one. However, we didn't really know this and hadn't properly set things up, so in one level that had a lot of waterfalls, about 75% of them just did not make any sound.
We spent a ton of time looking at the water code, and the audio code, and it was only after days of testing that we realized it was any sound after a certain number that didn't play. We then realized the issue was FMOD and not the code.
Another problem we had was the landing sound being slightly delayed. At some point, we noticed this had been happening for a few weeks. It was very slight, but definitely noticeable. You'd land, and then a split second later, you'd hear the sound play. It really took you out of the moment.
Similarly, we looked at the movement code, thinking the physics was off somehow. Finally after a few days, I started going through some old builds, and noticed that the builds a month ago didn't have this problem. I started going through old builds chronologically, one at a time, until I found the one where it started to happen. Since we have the changelist number in the name of the build, I then went to our perforce and reverted the project to each version. Again, no code changes during that time would have caused this.
Finally, we were led back to FMOD. The event for landing had been adjusted and there was a slightly longer track. It didn't seem like it would be a problem, but somehow it caused the sound to be delayed. Our theory is that because the track was longer than a certain length, it resulted in FMOD streaming in the sound.
On top of these problems due to subtleties in how FMOD works, we also had to implement a lot of debug tools in game to handle events, and to be able to turn sounds on and off. We also had to implement our own audio occlusion system, which I'll discuss in more detail in the next update.
We've since fixed many of these issues, and the game is sounding great, but it definitely has not been a stroll through the park.
Ironically, having gone through all this, we're now much more familiar with FMOD and would probably use it again. However, it's definitely not a tool I would recommend jumping blindly into. Unless someone on the team has a lot of experience with it, it might not be worth the additional effort.
Graphics IssuesThere are some graphics bugs we're trying to debug. One of the tools we're using lets us disable various effects at runtime.
Here are some cool screenshots that Aaron took:
Bug fixingBug fixing, bug fixing, bug fixing.
It honestly seems that is all we do these days. Fortunately we've got an amazing community of testers who've been finding all sorts of obscure bugs. And while it is a bit disheartening to see new ways the game breaks each day, we're really happy to be catching this stuff now and fixing it. The game is definitely much more stable than it was months ago.
Massive shoutout to vtech, cache, and californ1a who've been leading the charge on the bug finding effort.
Here's just a collection of some of the bugs:
Crack in the geo allowing player to sneak through:
Cube falling slowly for some reason:
Water falling through geo:
Jittery physics:
EndingFinally, for design, I'm working on the ending. Almost there. Honestly, the hardest part is just finding the time to be able to do design amidst everything else.
Community
