Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1297696 Posts in 57713 Topics- by 48763 Members - Latest Member: Sai Narayan

May 27, 2017, 10:51:04 pm

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsCommunityTownhallCosyne Synthesis Engine
Pages: [1] 2
Print
Author Topic: Cosyne Synthesis Engine  (Read 16089 times)
muku
Level 10
*****



View Profile WWW
« on: March 23, 2010, 05:13:01 pm »

Hey people,

I haven't been around these parts much lately for various reasons; maybe some people remember me anyway. However I've been working on something that may be of interest to those wanting to make a music or rhythm game, namely a lightweight realtime software synthesizer which can be integrated as a library. The thing is called Cosyne for the Cosyne Synthesis Engine (yeah, recursive acronyms rock). Instruments for the synth can be programmed in a special synth description language (documentation).

In this very first release, there's a demo application which lets you play with various synthesizers using your computer keyboard, and there are bindings to C and Python as well as some very basic example programs making use of those.

Download for Windows (check the readme):
http://bitbucket.org/eriatarka/cosyne/downloads/cosyne-bin-0.1.zip obsolete

Project homepage (source etc):
http://bitbucket.org/eriatarka/cosyne/wiki/Home

Some audio samples (this is only scratching the surface...):
http://www.box.net/shared/cflgzj8d26

I don't know if there's any interest for this kind of thing at all, so I'm just putting this out there to see what people think.

BTW, it's written in D.


=== UPDATE ===

The latest version of Cosyne can be obtained by going to the Downloads page and grabbing the newest binary package.
« Last Edit: April 06, 2010, 03:02:02 pm by muku » Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
John Nesky
Level 10
*****


aka shaktool


View Profile WWW
« Reply #1 on: March 23, 2010, 05:19:23 pm »

Hmm! I am making something similar! But I haven't had time to work on my thing lately, Cry so congrats on releasing this. I'll make a note to check this out as soon as I have time.
Logged
tomka
Level 0
**


View Profile Email
« Reply #2 on: March 23, 2010, 06:42:31 pm »

hmm ill see if i can get a os x binary up and running
Logged
Alex Vostrov
Level 3
***


View Profile WWW Email
« Reply #3 on: March 23, 2010, 07:20:19 pm »

This looks interesting.  Maybe now my games will have more-than-rudimentary sound.
Logged
muku
Level 10
*****



View Profile WWW
« Reply #4 on: March 24, 2010, 12:53:56 am »

hmm ill see if i can get a os x binary up and running

That would be too cool... do you have a D environment set up? I used DMD 1 with Tango. Let me know if I can help. The main issue is I don't know if DMD can build shared libraries on OS X. To get started, try the CosPlay demo application which doesn't do dynamic linking, it just compiles the library in.

Also I should mention the Bitbucket hg repository is a few commits behind my actual working directory because I had some trouble pushing to it last night. I hope this will be fixed soon.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
increpare
Guest
« Reply #5 on: March 27, 2010, 04:18:40 pm »

Hey muku, it's a very happy thing to see you pop back in here, even if just for a little.

I'll try this out either next time I'm back on windows or whenever the osx build gets built.
Logged
0rel
Level 4
****



View Profile WWW
« Reply #6 on: March 30, 2010, 06:04:38 am »

Muku, really nice to see you here again.

Such a cool project this is! I don't know why this thread is so quiet, really, it's quite amazing thing you got there, and it can be very useful for making games as well...

I played around with it a bit yesterday and made a small test program, with some modified instruments... It doesn't really sound so musical, but the sound quality is okay, and I really like the way how the instruments are described. It's clean and easy to understand, not as cryptic as other similar "synth toolkits". Still flexible and useful in practice, and extendable if really needed. Some documentation would help though. I for example didn't find out how to modulate certain parameters by LFOs/Envs. Some work, others don't... But I should check out the source files more in depth. They look really well organized and compact! Really well done.

Possibly useful features:
* Realtime control over parameters of the sound, like. (ChangeParameter(instrument,...) or something similar.)
* More than 2 oscillators per instrument?
* Polyphonic instruments? (PlayChord(...) or something similar... Multiple PlayNote()s with the same start time? Didn't try that yet...)

And to get the library working in C++, I had to change those macros in the cosyne.h file like this:
Code:
#ifdef BUILDLIB
#  define COSYNEFUNC(rettype, decl) extern __declspec(dllexport) rettype  __stdcall decl {}
#elif defined( __cplusplus )
#  define COSYNEFUNC(rettype, decl) extern "C" __declspec(dllimport) rettype  __stdcall decl ;
#else
#  define COSYNEFUNC(rettype, decl) extern __declspec(dllimport) rettype  __stdcall decl ;
#endif

Otherwise it worked fine immediately...

Still I'm not so sure if I used the right method to play notes interactively, after pressing a button. I tried it like this:

Code:
Cosyne_PlayNote( c, Cosyne_GetCurrentTime( c ), 20000, INSTR_XYZ, 40, 0x40 );

That worked for me, although the timing is a bit off that way, because OpenAL (and other Audio APIs too) only asks for buffer updates in certain intervals, so that PlayNote() events are only as accurate as the buffer size allows... But it doesn't sound so bad, and adds some "analog qualities" to it Wink.

(Probably there's a way around this though, by adding an offset manually, but I didn't try that yet... Instead of a buffer based render function, a sample based render function could also help with this maybe, although that would make the performance go down...)

By the way, I had an idea about MML (http://en.wikipedia.org/wiki/Music_Macro_Language), which found out about there a while ago... That's some crazy stuff for sure, and maybe the way it handles sequencing could be useful here too. I don't really understand how all this MML stuff works though, but it seems to be very powerful.. Maybe a separate library for sequencing only could be an other "module" of this toolkit, just an idea I had... Something like macros (for example for defining things like an arpeggiator, scales and rhythms), could be defined in seperate small scripts, which are generating note/parameter events, passing over to cosyne. Some parameters could be controlled interactively... It's a quite cloudy idea though... Don't now how it could be implemented myself Wink

I'd just love to edit instruments and possibly even "interactive sequences", while the final program keeps running... However, that's probably quite difficult with text based code files.

I also tried my hands on making a audio synth library (called 'executable sound'), you might remember it... It's still not fully finished though, but I'm working on it again soon. Mine isn't so "score" oriented, and it's harder to play notes and to define separate instruments the easily... And it can't be used in pure code/text, which makes things more difficult in practice. - This is one point that is really cool about your library. It's easy to make the most obvious thing: Play a note with an instrument! That's really useful.

Heh, this got me a bit inspired about realtime synths again. Thanks pal Smiley

The lib also works with MinGW, by the way...
Also, props for releasing it as opensource! Beer! Just hoping some people here will also try it out...
« Last Edit: March 30, 2010, 01:11:22 pm by 0rel » Logged
increpare
Guest
« Reply #7 on: March 30, 2010, 09:39:24 am »

This is neat!  Light a lightweight version of supercollider.

Would be cool to link to as a library, but I won't be able to do that unless there's a mac port done.

A direct 'output to wave' function would be cool - with that, I can imagine myself making plenty use of this for messing around/improvising.
Logged
Trunks7j
TIGBaby
*


View Profile Email
« Reply #8 on: March 31, 2010, 12:32:56 pm »

This sounds fantastic man!  I can't wait to check this out.  I just recently finished working on a small music game project, and throughout the entire process I kept thinking to myself that there is really a ton of room for improvement for a real-time synthesis library for programming languages.
Logged
agj
Level 10
*****



View Profile WWW
« Reply #9 on: April 01, 2010, 03:55:51 pm »

Hey! Cool to see you again. I really like the concept of real-time sound synthesis, and games that use that. I hope cool things come from this.
Logged

muku
Level 10
*****



View Profile WWW
« Reply #10 on: April 02, 2010, 06:16:13 am »

0rel, that's really neat, just the kind of thing I imagined people could do with it! Glad to see some people are interested in this. I just came back from a week in Berlin, so had no time to work on it, but as soon as I have some free time I'll attend to some of the issues raised here.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
JackieJay
Level 3
***



View Profile WWW
« Reply #11 on: April 02, 2010, 02:22:04 pm »

The name is just mind-blowing.  Epileptic
Logged

Tanner
Level 10
*****


MMPHM *GULP*


View Profile WWW
« Reply #12 on: April 02, 2010, 07:07:34 pm »

I don't think the name is the part that's intended to be mindblowing...
Logged

muku
Level 10
*****



View Profile WWW
« Reply #13 on: April 03, 2010, 01:59:38 am »

Ok, finally have some time. First of all, thanks for all the nice welcome-backs and such, it's appreciated Smiley

Some documentation would help though.
Yes, I should get to that ASAP.

Quote
* Realtime control over parameters of the sound, like. (ChangeParameter(instrument,...) or something similar.)
Yes, I've definitely thought about something like this, and it shouldn't be hard to implement. I guess I'll do this soon.

Quote
* More than 2 oscillators per instrument?
Would certainly be attractive. So far, I stuck to only 2 for several reasons. First of all, the plan was to keep synthesis cheap enough to run in the background of a game which does a fair amount of processing itself. Besides, the 2 oscillators plus filter layout is sort of a classic design of (soft)synths, so it's immediately familiar to anyone who has done some synth programming. (In fact, if you know the synth1 VST, I've copied a lot of its layout.) Finally, having a variable number of oscillators would probably necessitate a more modular design which could complicate things, both internally and for the user. On the other hand, just adding an optional third oscillator which gets mixed in with the others would be more or less a triviality to add, so that's definitely an option if people want it.

Quote
* Polyphonic instruments? (PlayChord(...) or something similar... Multiple PlayNote()s with the same start time? Didn't try that yet...)
Yep, multiple PlayNote()s do work. Just watch the volume so that you don't get clipping. I'm not sure yet how to deal with this whole volume business properly. Anyway, you can try it in the CosPlay sample application, just hit several keys at once.

Quote
And to get the library working in C++, I had to change those macros in the cosyne.h file like this:
Ah, thanks for this, forgot to try it in C++. Will update the header file accordingly.


Quote
Still I'm not so sure if I used the right method to play notes interactively, after pressing a button. I tried it like this:

Code:
Cosyne_PlayNote( c, Cosyne_GetCurrentTime( c ), 20000, INSTR_XYZ, 40, 0x40 );

That worked for me, although the timing is a bit off that way, because OpenAL (and other Audio APIs too) only asks for buffer updates in certain intervals, so that PlayNote() events are only as accurate as the buffer size allows... But it doesn't sound so bad, and adds some "analog qualities" to it Wink.

(Probably there's a way around this though, by adding an offset manually, but I didn't try that yet... Instead of a buffer based render function, a sample based render function could also help with this maybe, although that would make the performance go down...)

The way you tried was fine. (As a shorthand, you can use -1 instead of GetCurrentTime(), or maybe 0xFFFFFFFF if the compiler complains about signed-unsigned mismatch.)

Cosyne internally works with chunks of 64 samples instead of processing each sample individually to keep the CPU load manageable. If each chunk could be output immediately without delay, at a sample rate of 44.1kHz, this would correspond to a latency of 64/44100 = 1.45ms, which is basically realtime as far as humans are concerned. So this design shouldn't be a problem.

The problem is how to get the data to your soundcard as quickly as possible. As you mentioned, all audio APIs have some kind of cyclic buffer for audio data. The key point is the size of this buffer, which the API should let you choose upon initialization. If it's too large, you will get bad latency, if it's too small, audio will crackle and skip. This is to some degree hardware dependent, so I guess you should offer it as an option to your user, but in general I've found that 512 samples is an acceptable compromise.

Keep in mind that a game running at 60fps corresponds to having 735 samples per frame (at 44.1khz), so a 512 samples buffer should be just fine to keep perfect sync with the video.

Quote
By the way, I had an idea about MML (http://en.wikipedia.org/wiki/Music_Macro_Language), which found out about there a while ago... That's some crazy stuff for sure, and maybe the way it handles sequencing could be useful here too. I don't really understand how all this MML stuff works though, but it seems to be very powerful.. Maybe a separate library for sequencing only could be an other "module" of this toolkit, just an idea I had... Something like macros (for example for defining things like an arpeggiator, scales and rhythms), could be defined in seperate small scripts, which are generating note/parameter events, passing over to cosyne. Some parameters could be controlled interactively... It's a quite cloudy idea though... Don't now how it could be implemented myself Wink
Yes, that's very interesting stuff. I also did some stuff with procedural composition in Python a while back which I could now hook up to Cosyne (maybe you can still find the thread here). In general, I think these sorts of things should be implemented on top of Cosyne, not within it: I've decided to keep the engine closely focused on efficient generation of audio data. This is why the score functionality is so rudimentary and doesn't even offer something like a beats per minute setting, just raw sample counts. (Though maybe seconds would be nicer.)

Quote
I'd just love to edit instruments and possibly even "interactive sequences", while the final program keeps running... However, that's probably quite difficult with text based code files.
A very nice thought, and I think not even that hard to implement. Just check if the file has been modified (usually the OS will give you some kind of hook to listen for this, Windows does at least), and if so, reload it.

Quote
I also tried my hands on making a audio synth library (called 'executable sound'), you might remember it... It's still not fully finished though, but I'm working on it again soon. Mine isn't so "score" oriented, and it's harder to play notes and to define separate instruments the easily... And it can't be used in pure code/text, which makes things more difficult in practice.
Yes, I remember it, it was included with that 3D flying game of yours, right? (Occuplector or something like that, IIRC.) It was very cool, though it took a long while to render. Would definitely love to see any progress on that.

Quote
- This is one point that is really cool about your library. It's easy to make the most obvious thing: Play a note with an instrument! That's really useful.
Yes, ease of use (and binding to other languages) was a goal with this.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
muku
Level 10
*****



View Profile WWW
« Reply #14 on: April 03, 2010, 02:19:54 am »

This is neat!  Light a lightweight version of supercollider.

Would be cool to link to as a library, but I won't be able to do that unless there's a mac port done.

Yeah, sorry about that. There's nothing really platform-specific about it, so a port shouldn't even be hard to do, I just don't have access to a Mac Sad

Quote
A direct 'output to wave' function would be cool - with that, I can imagine myself making plenty use of this for messing around/improvising.
What are you referring to? The library itself is totally backend-agnostic, it just generates samples, what you do with them is up to you. For convenience, a simple SDL backend is provided. The CosPlay app actually has a command line switch to record to a wav file. Or maybe you were referring to 0rel's thing.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
muku
Level 10
*****



View Profile WWW
« Reply #15 on: April 03, 2010, 02:21:57 am »

The name is just mind-blowing.  Epileptic

Heh. It must have been a rare flash of inspiration Wink
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
increpare
Guest
« Reply #16 on: April 03, 2010, 04:28:00 am »

The CosPlay app actually has a command line switch to record to a wav file. Or maybe you were referring to 0rel's thing.
Oh - I didn't know that!  Cool.
Logged
0rel
Level 4
****



View Profile WWW
« Reply #17 on: April 05, 2010, 08:34:11 pm »

Quote from: muku
Yep, multiple PlayNote()s do work.
Ah that's great.

Quote
Just watch the volume so that you don't get clipping. I'm not sure yet how to deal with this whole volume business properly.
Hm, that volume issue was annoying me too by the way. I also don't have a perfect solution for this though...

Quote
Keep in mind that a game running at 60fps corresponds to having 735 samples per frame (at 44.1khz), so a 512 samples buffer should be just fine to keep perfect sync with the video.
Another solution that came to mind was to handle the interaction at a different rate than the audio API asks for updates, which can be unpredictable at times (OpenAL uses somewhat mysterious buffer queues for example). I often use a Move() function in my game code separated from Render() to handle all interactions/animations/state changes, which should be independent of the framerate. So, Cosyne_Render() could be called there at steady intervals, to render a small number of samples to a temporary soundbuffer, which would be cleared at the next audio API callback... That way the interaction (PlayNote/ChangeParam events) would affect the audio at a much finer resolution, and one wouldn't directly depend on the audio buffer size. Maybe I'll try that out this way... It isn't really a issue of your library though, more about how to use it.

Quote
I also did some stuff with procedural composition in Python a while back which I could now hook up to Cosyne (maybe you can still find the thread here).
I think I remember those... Really interesting things there! Besides the difficulties about the micro-scale sound programming, the actual composition is even more complicated to "get right" in my experience. Needs so much trial-and-error testing and knowledge about music theory (which I sadly don't have)... To have compositions like these now as Cosyne programs would be definitely cool... and inspiring too Smiley

@MML:
Quote
In general, I think these sorts of things should be implemented on top of Cosyne, not within it: I've decided to keep the engine closely focused on efficient generation of audio data.
I concur, everything else could be plugged-in later on.

Quote
Yes, I remember it, it was included with that 3D flying game of yours, right? (Occuplector or something like that, IIRC.) It was very cool, though it took a long while to render. Would definitely love to see any progress on that.
Ye, okkuplektor... it took ages to prerender. Those sounds were quite expensive though. The library itself is basically able to do sounds at an interactive rate (I already used it in my other games that way), with accordingly simple instruments of course. But it is still generally more about sound programming than about instruments. Otherwise it's quite similar to yours, also not depended on a specific backend/API, and not too complicated to use. But I really have to complete the documentation before anything can be released (if ever, publicly), otherwise it's really difficult to get started. Some parts about the patch editor aren't as obvious and self-explanatory as they should be...
Logged
muku
Level 10
*****



View Profile WWW
« Reply #18 on: April 05, 2010, 11:47:48 pm »

Thanks, some interesting points there. I'm at work now and will reply in more depth later. Just wanted to point out that I've done some documentation on the synth language. It's not complete yet, but should give people a start.

Tonight I'll probably do a new release with parameter changes implemented and some other smaller fixes.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
muku
Level 10
*****



View Profile WWW
« Reply #19 on: April 06, 2010, 02:56:46 pm »

Ok, Cosyne v0.2 is out. We now have user parameters as suggested by 0rel, and CosPlay can modulate them with the mouse if you pass it the -m switch. There are some other minor changes, mostly better error reporting.

Also, the first draft of the synth language docs is now more or less complete.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic