Environmental AudioIn contrast (
) to the low definition visuals, I'd like the audio in this game to be pretty high def. That means using realistic sounds and mixing for everything. Ideally the player could close their eyes and forget they're playing a black and white low resolution indie game.
SoundsLuckily for me there's a solid set of iconic environmental sounds associated with wooden sailing ships:
- Wind
- Flapping sails
- Stretching rope
- Lapping water
- Creaking wood
- Woody footsteps
Putting those together in a static mix gets you a nice above-decks scene. The hard part is dynamically positioning and mixing them all together to make the ship seem real as you move through it in first-person.
MixingThere are a couple problems with trying to do complex ambient sounds in a 3D game. First is that basic 3D positional sounds aren't that useful. Most ambient audio doesn't emit in a sphere shape from a central point. Or if it does, then you need a lot of point sources scattered around to get the right sound.
Second problem is how to handle positional modulation at all. Because ambient sound comes from all around, you'd want that to modulate as the player turns their head. The proper solution is 5.1 surround - where the position information covers the entire 360 degrees and not just L/R.
So I just need to find a source for 5.1 ambient sounds.
Sourcing AudioFor the last few years I've been using Freesound.org as the primary source for audio in my games. The files there are rarely usable as-is, but with a little tweaking and combining you can get good stuff. Freesound doesn't have a lot of coverage though so I've also used SoundSnap.com (paid) to fill in the blanks.
I went back to those sites for this game and found them almost completely lacking. Very few useful footsteps, no usable lapping water, stretching rope, sails, etc. Kinda surprising.
I asked one of my pro audio designer friends where they get these kinds of sounds when they need them and his answer was: "We hit our back catalog, record them ourselves, or order them from a foley professional." Hmm... Well that
would be cool, but I don't think I'll go that far. These aren't the kind of unique sounds that need custom recording. I just need to find a good source.
After a bunch of searching (All sound effect websites are straight out of 2001), I came up with 2 good ones: Sounddogs.com and Pond5.com. Both of these are expensive from the perspective of an indie developer. Based on the success of Papers Please though, I have more resources to use money in order to save time like this. So instead of searching for hours and hours for just the right free/cheap sound then processing it to fix things up like I used to, I'm just finding the right paid sound, buying it, and doing less processing. I still don't have a lot of expenses (no big team, no office) so it's really not much in the end. I think around $300 for all the sounds I'll need.
Anyways, one potential problem is that all paid sound effect sites only give you a low-fidelity preview of the audio. Already a few of the clips I've bought have turned out to have problems that were inaudible in the preview.
After finding exactly zero 5.1 sounds, I gave up on getting surround sources and settled for stereo.
EnvironmentsSo now I've got stereo audio sources and I don't want to use 3D sound spheres. How to position the audio? The first thing is to consider the environment. It's a smallish ship with 4 decks stacked vertically. Only the topmost deck is open, and there's a set of cabins at the rear. If I break it down:
- Top deck - open
- Aft cabins - closed
- Gun deck - closed roof/walls, open gun portals, near waterline
- Lower deck - closed, partially underwater
- Cargo deck - closed, underwater
Each of these has a different ambient sound to them, but what's really important is the transitions: How the ambience changes when going into the rear cabins on the top deck, for instance.
Sound RoomsIn order to get all these ambient sounds going in each different environment I created a fairly simple component called a "SoundRoom":
Boom. SoundRoom.
The goal is to be able to position the ambient sound around the player wherever they are. Instead of playing individual 3D sounds, I play all environmental sounds on a loop in 2D and manually adjust their volume & pan as the player moves around. SoundRooms let me specify how that should be done based on the position and orientation of the audio listener.
Some games use environmental modeling with raycasts or geometry but I figured the ship was simple enough that boxes would be enough. I'm sure this technique has been used in lots of games. It's simple and it works well.
For each SoundRoom, I specify the audio clip that it modulates along with the properties of the 6 walls. Each wall has a setting to define how it affects the pan and volume of an ambient sound as the player moves around inside:
- Solid: No affect on the sound inside the room
- DirectionalSource: This wall emits the sound
- DirectionalDestination: This wall receives the sound
- DirectionalPass: This wall affects only volume and not position
- DirectionalRoom: This wall uses another room to calculate position/pan
- AmbientSource: This wall has pan=0 (to transition from open ambient to directional)
Each of these types was added by trial and error as I built out the SoundRooms and needed different features for directing the sound.
When laying out the soundrooms, I set each wall to whatever type will direct the flow of sound through it in a realistic way. During gameplay as the player's position inside the room changes, I interpolate between the properties on each wall to generate the final volume and pan for each ambient sound affected by the room.
Wall InterpolationOne interesting challenge that came up was how to actually interpolate between the volume and pan settings for the surrounding walls. The normal thing you use when dealing with 2D stuff is basic bilinear interpolation.
Bilinear interpolation
Unfortunately, bilinear interpolation interpolates between the
corners. What I want is to interpolate between the
edges. Trying to figure out what to even search for was a little tough, but I eventually found something called
inverse distance weighting which did the trick.
Inverse distance weighting
Inverse distance weighting is normally used to interpolate between discrete points but it also works in any case where you can calculate a distance between two things - Like between a point representing the player's position and an edge representing a wall.
Implementing the solid walls is as simple as ignoring one edge's contribution:
The right side is a solid wall, ignored for the interpolation
One big advantage of this setup is that I can dynamically change the wall types. So the ambient sounds can change realistically when doors open or close to let in or block out different neighboring environments.
Editing in UnityI've been constantly surprised with how easy Unity makes things. When laying out the SoundRooms, it's useful to have in-editor features like wall snapping, duplicate+flip, sound flow visualization, and more. All of this was trivial to implement in Unity's scene editor.
Sound flow visualization.
The system they have for seamlessly adding features to the editor is really great. As a former tools designer I'm often impressed by what's not only possible, but easy to do in Unity.
Spheres Too Why NotFairly quickly I found a case where elongated spheres
would work better than boxy soundrooms. For those, I added a SoundSphere component that implements the same basic systems as the SoundRoom but uses a scalable sphere for the area.
Wind+water sounds through the gun portals
As mentioned, I found the key to making the ambience feel right was to get the transitions right. Rooms where the audio changes from being all around to coming from just one direction sound great.
The other big win is with changing the audible frequencies as the player moves between open air and interior spaces. The technical term is
rolloff I think. Basically, given an ambient sound, the high frequencies are more directional and can't penetrate walls. The lower frequences have less directionality and can be heard through walls.
This means that as you step into the cabins for instance, the high frequency portion of the wind and wave sounds should trail off, leaving the low frequency rumbles. What's really nice is when, during this transition, the high frequency part maintains its directionality - "this sound is coming through the door to the outside, which is to my right."
After some experimenting, I found that it doesn't take much to nail this effect. At the moment, I have only one ambient sound that does this rolloff - the main wind+waves loop. All other ambients are localized enough that simply fading them in/out is enough.
Unity includes an AudioFilter that can be used to apply a highpass or lowpass filter. But because I also want to split the positional handling of each portion, I need at least two sounds playing anyways. It makes more sense to just pre-process the original audio file to split it into separate "-Lo" and "-Hi" parts, which can then be played/panned separately without expensive AudioFilters.
FootstepsThis is a first person game so aside from the ambient sounds, there's a constant beat of footsteps as the player moves around. Changing these up to represent different surfaces really helps to make the ship feel more real.
It's a pretty small ship area-wise so I don't need too many footstep variations. So far I've got normal, grate, stairs, and carpeted footstep sounds. To specify which surfaces make which sounds I'm using a simple derivative of the SoundRoom: the Zone:
A zone for setting footstep sounds to "grate"
I think most games associate footstep sounds with textures or geometry. To determine which sound to play, you cast a ray down from the player, see which texture it hits, then look into a mapping of texture names to sound names.
This game has so few textures though, and so few different surface types, that I decided it'd be easier just to create little oriented boxes around the surfaces to specify which non-default footsteps to play. At runtime, I test the foot position to see if it's inside any zone and if so, use that footstep sound. Along with the footstep I also randomly play a light creaking sound every once in a while on foot down.
All of em
Since moving my development to OSX ~5 years ago, I've used
Audacity for editing audio files. It's a great program, easy to use, multiplatform and with lots of features. The only downsides are that it's slow and all of the edits are destructive. Making lots of little tweaks or testing different things is harder than it should be.
I was all set with my typical workflow in Audacity when I started to think about alternatives. As mentioned above, I have a little more money to work with on this project. I like that Audacity is free but I'll bet there's more efficient tools out there if I'm willing to pay for them.
Sure enough, Adobe's Audition fits the bill perfectly. It's part of Adobe Creative Cloud, so the "rental" thing is a little off-putting. But the features and workflow are so good that I think it's worth it. In particular the sound and reverb removal features have already saved my ass.
Some ExamplesA few places to show the SoundRooms in action.
Two of the aft cabins. High frequency wind+waves sounds
can enter through the door at the left or the window at the
bottom right. At the top right, the blue circle represents a
positional transition to pan=0 as the sound fades to the
main room, where it comes from all directions.
Stairs connecting the top deck with the gun deck.
Multiple soundrooms are used to fade out the high
frequency wind+waves audio as you descend the stairs.
I experimented briefly with Unity's AudioReverb component but couldn't find anything that sounded very good. I think it would be nice to have interior reflections when you go below decks though so I'll probably go back to this at some point and try again.
Walkthrough I recorded a short walk through the different environments. No visuals, just audio. After all that work it sounds pretty average really. I feel sorry for sound designers. A lot of effort to create and place sounds just so the player hears what they expect and doesn't notice anything unusual.
Community
