next »

[Evan Balster]

Once again, the time has come for...

         __        _       _     _    __              _ _        __
        / /  _ __ | | __ _(_) __| |  / /_ _ _   _  __| (_) ___   \ \
       / /  | '_ \| |/ _` | |/ _` | / / _` | | | |/ _` | |/ _ \   \ \
       \ \  | |_) | | (_| | | (_| |/ / (_| | |_| | (_| | | (_) |  / /
        \_\ | .__/|_|\__,_|_|\__,_/_/ \__,_|\__,_|\__,_|_|\___/  /_/
            |_|
                        standalone audio framework
                              Version 0.2.0
                       Written with <3 by Evan Balster


<plaid/audio> is a portable, extensible C++ framework for CPU-processed audio.  It is designed chiefly for use in games and other media-intensive software, and is most notably used in SoundSelf.

It offers a free (zlib-licensed) alternative to commercial audio middleware like FMOD.  Unlike the leading API, OpenAL, its behavior is consistent and guaranteed across platforms.

At present it offers classes for streaming and buffering audio files, playing back microphone input, manipulating streams with a selection of simple effects including pitch-changing, bandpassing and amplification, as well as mixing and splicing.

plaid/audio's scheduling system synchronizes audio with the program's framerate.  It does this by dividing output into "timeslices" for each call to update().  This allows games to render their audio at their graphical framerate, with any changes in audio effects, synthesizers or samplers being audible every frame.  Physics, animation or player input can thus have articulate audio feedback.


Extensive documentation is now available here.


A Visual C++ example project is included, which plays a little song.  It shouldn't be hard to build for other platforms; the audio engine code has previously been compiled for Windows, OS X, Linux and iOS, and the provided implementation layer (portaudio) is compatible with all but the latter.

Special thanks go to TIGS user randomshade of Pixelscopic, who hired me for the work of separating this code from my game engine, with the understanding that it would be opensourced thereafter.


Download:
plaidaudio_0.2.0.zip 588kb (alpha)
Demo Program (Windows) 237kb

Backwards code compatibility is not guaranteed with future versions.


Notable inconveniences at present:
- The only codec I've written is for OGG files.  Further, it uses stb_vorbis, which can be a little picky about its input.  If you control the files you use this isn't a problem.  I'll accept codec contributions if they don't add library dependencies.
- The Audio module expects wide strings for the purpose of specifying audio files.  (My game engine is unicode, but yours probably isn't)

[noah!]

Wow, I've been looking through it, and this looks beautiful. It's like the sound library I've always wanted to make, only this one is programmed way more competently than anything I could do...

I look forward to getting some time to play with this, and I'm looking forward to seeing where you go with the library!

[Belimoth]

Impressive.

[eclectocrat]

Super cool, will play around with it.

[Graham-]

Are the main values: free and consistent?

[Evan Balster]

Free, consistent, powerful, optimized for games.  With time I expect it to compare favorably with commercial libraries like FMOD.


I plan on making a second release soon.  I've fixed the Ref pointers being a pain in the butt and changed the MSVC project so it builds a static library.  If nothing else I should pack up those changes into a 0.1.1.  I would also like to write some decent documentation and provide more interesting example programs, but that might take a bit longer.  I might give the former a start tomorrow.

I'm super interested in any feedback anyone who's messed with this has.  Also I'd be interested to see anything that is made with the engine.

[Graham-]

Is FMOD slow? I have little (0) experience w/ these libraries.

[Netsu]

Sounds really cool Does it have positional sound? Might be interested in switching my game from my own OpenAL wrapper lib to this if I finally get around to finishing it. How big is the compiled lib?

[Evan Balster]

Graham:  FMOD is great, it's just expensive.  This will be a free, more readily extensible alternative.

Netsu:  It doesn't have 3D sound, just a smooth-ramped pan filter.  (On some OSes all OpenAL's 3D positioning does is pan and amplification!)  It does support arbitrary output channels and writing a DSP for dolby 5.1 based on the Pan filter would be entirely possible.

The Visual Studio lib appears to be 13 MB, but a minimal EXE built with the thing (plus the provided implementation layer and OGG codec) is just just 234 KB.  The library is pure C++ and doesn't require anything except an audio implementation layer and optionally some file codecs, both of which I've provided.  On that note I should add my SDL audio implementation layer to the next release.

[Sergi]

This looks pretty awesome!

I have no idea whether this is possible or not: is there any way of using a library like this from Unity? Like, a compiled dll that can be written in any language, or something like that (I guess that would break portability)?

[Evan Balster]

It's possible.  FRACT uses PD as its sound engine, interfacing with it through what I imagine to be DLLs.  You'd need to build the extensions for every target system and it would probably break compatibility with the web player.

My understanding, though, was that Unity's sound engine was FMOD, which was pretty full-featured as I recall.  Then again, you can write whatever kinds of audio effects you want with my system so that's always nice. 

[Graham-]

Ah. Maybe I'll learn something from it.

[Sergi]

My understanding, though, was that Unity's sound engine was FMOD, which was pretty full-featured as I recall.  Then again, you can write whatever kinds of audio effects you want with my system so that's always nice. 

Oh, I actually didn't know much about Unity's sound engine. My knowledge is just "play music, play sound effect", but I just checked the docs and it seems to have a lot of stuff, including some effects with the paid version. Cool.

[Gregg Williams]

We might give this a shot for our next major project. Kinda tired of fighting with the OpenAL platform inconsistances. Though we've also been considering irrklang.

[Evan Balster]

I've discovered two high-priority bugs in the system which will be fixed in the next release.

The first is triggered by loading data into AudioClip objects at startup and causes multithread-related memory corruption in the Audio system's scratch-pool allocator.

The second is triggered by dropping a stream from a Mixer which has been denied scratch memory by the aforesaid allocator, and causes the audio callback to infinite-loop, creating a deadlock.  On one occasion this caused the Windows audio kernel to crash on me!  D:

Expect a new version with bugfixes, improvements and some documentation sometime this week.

[Triplefox]

Hey, audio buddy. I've been doing a lot of DSP study while working on my Soundfont/SMF player over the past year so I'm always looking at other people's code for ideas now. I'm going to be overhauling my own stuff soon to emulate the HTML5 Web Audio spec, rather than my current overly-coupled thingy, which grew out of sequencing and sample playback, not the signal chain.

I browsed through the Plaid source for a while and noticed some nitpicky stuff:


Panning commonly uses an "equal-power" or "constant-power" crossfade defined by the sin and cos of the pan value (times pi/2), rather than a linear fade. This is because linear weakens the output at the center values. There are reasons to use both kinds of fade though.

The percentage amplitude of dB is defined with 10^(db/20). It's the "power ratio" that uses 10^(db/10). I've only just sorted out the terminology difference, but I've been using the first function with the Soundfont spec for a while, after going through many other functions that were wrong and bad.

It's possible to use a higher order Hermite curve for resampling interpolation before implementing sinc, and I do this for mipmapping. I refer to this paper for the implementation. (I don't recommend actually using the optimal polynomials described. They require oversampling and filtering to not sound horrific, so the results are nearly useless as a cheap interpolation strategy.)

I see no cutoffs for very low (under -96dB) amplitudes or noise mixing, which is bad because going too close to 0 will make the floats denormal and cause a massive CPU spike. (I need to do this for my reverb still. It has the nasty habit of freezing the process during quiet moments.)


Mostly, Plaid looks pretty good. I'd use it, given the right project. It's quite close to what the Web Audio guys are doing, too, which is a good sign. Rock on

[Evan Balster]

Yes!  Some feedback for once!  Thank you.  <3


Pan:  I will make that change immediately.  The scheme was bugging me but I didn't really think it over much.

Decibels:  Ah!  Thank you for setting me straight there; I'll fix it straightaway.

Resampling:  I'm well-acquainted with the pink elephant paper (see my bookmarks) and I really like it.  I've been pretty satisfied with four-point resampling in terms of sound and speed.

Cutoffs:  I don't use floats.  My audio engine was originally going to support multiple sample formats, but writing every processor for all those different formats was a pain.  In the end I decided on 24-bit fixed-point sound stored in 32-bit signed integers.  This is fast, permits a huge dynamic range and facilitates safe overflow that can be managed with limiters.  I also don't have to write clipping logic into every little thing like I did with 16-bit...

I'm thinking I should also add some higher-order lowpass/highpass filters.  I'd like to make Bandpass use Butterworth filtering of user-defined order but haven't been able to find a good code reference for that.


Status:  I've been working on extensive documentation covering the next version (sneak peek) and fixing the major bugs I mentioned previously.  I was hoping to have a version 0.2.0 out for Christmas (because Christmas) but the new year is perhaps a more viable goal.  After that release I'll try to do patches on a more nitpicky, frequent basis.

[Overkill]

Hey, Cellulose! I just tried this, and successfully integrated it with my project! It seems pretty rad, good work! And thank you for providing it under a zlib license, that was super awesome, and saved me the mind-imploding headache of writing my own permissive-licensing friendly audio library, something I know next to nothing about. I thanked you on Twitter earlier, but I finally got around to playing with it.

I even managed to make a new codec without many problems, so I could play mod/s3m/it/xm with libmodplug, and it was super easy! It was a bit hairy to convert an interleaved libmodplug buffer into the non-interleaved block that AudioChunk provides, but thankfully not tremendously bad.

I'll probably do the same for wav loading (I have silly parser lying around), and swap stb_vorbis for libvorbisfile for conformant .ogg playback.

I noticed the performance and audio quality really degrades if too many samples play concurrently. Is there anyway to make the clipping + weird distortion + audio lag less severe when several stream playbacks all stack together? I'm guessing this needs a different scheduler implementation, which actually uses threading for stream buffering? I don't know the first thing about improving that sort of stuff though, I just know the quality currently suffers compared to Audiere, which I was using before for playback purposes despite being LGPL and like a decade old.

I can send you a sample build of my silly demo game and point you at engine source if it's any help testing.

Also! There is a lot of std::cout spam. Any chance that could be conditionally compiled?

[Evan Balster]

Hello!  No shortage of familiar faces today.

Non-interleaved buffers:  Just to explain my motivation for this, it's *really nice* to be able to generalize audio effects like pitch/amp/bandpass to an arbitrary number of channels.  So it's a mercy towards writers of audio streams that makes life a touch harder for implementation and codec writers.

Codecs:  I'll be very interested to see what you make.  Note that I've already written a conformant ogg codec; it's macro'd out in the stb_vorbis codec and you could make a converted copy.  If your wav codec is pure C++ I'll also be very interested to see that.

Performance with many streams:  Streaming sounds straight from the disk is can be costly; layering more than a handful of streams can cause disk lag with severity proportional to disk bandwidth.  The solution is loading them into memory with AudioClip, which is unfortunately extremely dangerous to use in the current version of the library.  The workaround is to instantiate a "headless" audio module and pass it to AudioClip's load function when you use it.  This will be fixed in 0.2.0

Distortion with many streams:  Probably you just need to amp them down a bit.  In the future, I'll be adding a Limiter class and maybe an "effect chain" mechanism which could be used to place one between the master mixer and audio output.  I might even make it so a Limiter is in there by default...

std::cout:  Good point, I can have a look at that.  Notably, though, portaudio's debug lib spits a bunch of data to stdout.

I would be interested in seeing what you've made; mail or message me if you like.

[Klaim]

It is a very interesting initiative and I can only encourage you to continue with this! 

Quickly taking a look at the code, the interface seems nice and flexible. I don't like the `new` calls in the test and wish you had used something like std::make_shared to both benefit from performance enhancement and exception guarantees.
I suppose you didn't use standard smart pointers because you don't want to for the user to use C++11?

What I don't understand is that you use copies of object in most interface, yet you provide a reference counting mechanism (which is not used in the copy interfaces). Am I missing something?