Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

407879 Posts in 13860 Topics- by 26277 Members - Latest Member: gonna24you

July 31, 2010, 06:04:28 PM
TIGSource ForumsDeveloperTechnicalProcedural music generation: a hands-on experiment
Pages: [1] 2
Print
Author Topic: Procedural music generation: a hands-on experiment  (Read 5837 times)
muku
Level 10
*****



View Profile WWW
« on: July 12, 2008, 06:53:30 PM »

Hey all. After two days of hacking away at it (two nights, more like), may I present to you my craptacular procedural improviser getting funky:

http://www.box.net/shared/b2kvacxgco

It's almost 5am here, so... more info tomorrow, if anyone wants it.



UPDATE: Here's a link to a flash widget where you can directly play (without downloading) all mp3s I've created so far.
http://www.box.net/static/flash/box_explorer.swf?widgetHash=yiivqi0w0o&cl=0
« Last Edit: July 13, 2008, 06:24:18 AM by muku » Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
imaginationac
Level 1
*


Makin' games instead of makin' babies.


View Profile WWW Email
« Reply #1 on: July 12, 2008, 08:46:09 PM »

Hmm...interesting. Has a jazzy feel too it.

I'll be waiting to hear how you made this Smiley
Logged

J
Level 0
**


'Gotta roll with it...


View Profile
« Reply #2 on: July 12, 2008, 10:30:18 PM »

Procedurally generated music, that's something i haven't heard a lot about (actually, haven't heard about it period outside of Spore), but a very interesting concept. Song actually sounds good too (I came in here with the assumption that the notes would be all over the place), and like imaginationac said, it does hold a jazzy feel to it.
Very well done, interested to know how it was created.
Logged

muku
Level 10
*****



View Profile WWW
« Reply #3 on: July 13, 2008, 03:07:52 AM »

Ah, a good 7 hours worth of sleep, nothing quite like it Wink

Well, I've been working on a very procedural-content-heavy concept (main inspirations: Dwarf Fortress and Spore) for months, on and off, and I've always had vague fantasies how complementing that with a procedural soundtrack would be very cool. Well, a few days ago I finally got curious enough to actually give the idea a try; I figured I had the music theoretical background basically down (I dabble in jazz guitar, you see), so I got hacking.

First things first, choice of language. I played around in puredata for a while, and it's a really cool environment with tremendous possibilities, but me, I'm simply much quicker writing good old-fashioned code instead of dragging boxes around. So, a "real" language. While my main project is in D (which I can only recommend, it's such a boost of productivity over C++), I figured for this extremely explorative experiment something even more dynamic would be a good match, so I settled for Python. The immediate edit-run cycle turned out to be invaluable for testing, as compared to the edit-compile-run of statically compiled languages. Of course, should this ever get to a really usable state, I would reimplement it in a more static language.

Second, I needed a way to actually coax notes out of the speakers. Perhaps there are free softsynth libraries which can do this easily (I'm curious, are there?), but the easiest way to go seemed to be MIDI. In reality, it turned out there's a surprising scarcity of non-bitrotted MIDI libraries in Python (for a language which otherwise seems to have libraries for everything and the kitchen sink), so in the end I settled for a combination of the C portmidi library (which I had to compile myself too, what a hassle) and a crappy ctypes wrapper called portmidizero around that for the Python side. I actually had to fix bugs in that wrapper as I went as it would sometimes crash randomly, but eventually I was able to output notes conveniently to the soundcard and I was good to go. Of course, the basic cheap-soundcard-MIDI sound is awful, but hey, it's an experiment.

So next, the really interesting stuff, the music. I started teaching Python about scales, so my program knows about the major and minor scale as well as the modes of the major scale, i.e. Dorian, Mixolydian, that sort of stuff (if you know a bit about music theory, that should sound familiar).
Then I taught it to form simple seventh chords over a scale by just doing basic thirds stacking, and finally I implemented chord progressions so I could do stuff like a I-IV-V-I progression. I actually did some experiments with that particular progression, but it just sounded awfully stilted and formulaic, so for now I've settled with a more modal approach where I mostly harp on one root chord.

Somewhere along the way I implemented a simple pseudo-walking bass which would simply play chord notes on every beat as well as sometimes chord extensions in between, which actually sounds pretty nice for a first experiment.

As for rhythm, I started with a class which would accept patterns in a form like 'XOXO' and translate that into proper timings. That turned out to be a great concept as I could then very easily write algorithms to generate patterns or create variations on them as they are simply strings.

There are no drumloops so far, but with this rhythm system in place (which is used for all instruments so far), it shouldn't be hard to do something basic.

The last thing I've done so far is the solo-type melody you hear in the mp3, and that is basically a very simply Markov process for the notes with some more or less randomly chosen note durations, with care taken that it realigns with the beat every once in a while so that it doesn't sound too random. The jazzy sound of it might on one hand come from this whole jazz theory thing I've put into it; on the other hand, it actually seems simpler to me to randomly create the kind of jagged, syncopated lines characteristic of jazz/funk improvisation as compared to simpler, more catchy melodies. Though I definitely want to try the latter. The current form of "melody" would get on the player's nerves within minutes, so I need something more subdued as well.

At the moment, the structure of it all is very hardcoded. There probably is a wide range of stylistic variations possible already (I haven't even played with the parameters all that much), but it's not readily accessible. The ultimate goal would be to make this into a (open-source) library which everyone could integrate into their game; ideally, you would simply give it parameters like levels of "danger", "calmness", "excitement" etc and it would generate a soundtrack in realtime for you. That's of course a long way off. But it's nice to dream, isn't it? Wink

Well, that turned out much longer than I hoped. Sorry, I've got an annoying tendency to ramble.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
J
Level 0
**


'Gotta roll with it...


View Profile
« Reply #4 on: July 13, 2008, 05:33:57 AM »

Now that was quite music jargon heavy, but i understand the basic idea of what's going on. Great to see how much thought and effort you've actually put into this thus far.

The ultimate goal would be to make this into a (open-source) library which everyone could integrate into their game; ideally, you would simply give it parameters like levels of "danger", "calmness", "excitement" etc and it would generate a soundtrack in realtime for you. That's of course a long way off. But it's nice to dream, isn't it? Wink
That would have to be one of the best idea's i've heard in a long time, i'm sure you just got the support of every indie developer, especially those who've had trouble in the past finding music composers for their games.

Well, that turned out much longer than I hoped. Sorry, I've got an annoying tendency to ramble.
'Contrary, was quite an interesting and enjoyable read Wink
Logged

muku
Level 10
*****



View Profile WWW
« Reply #5 on: July 13, 2008, 06:12:39 AM »

Turns out this thing is insanely fun to play around with once you start tweaking stuff. I've just written a new melody generation algorithm which produces more laid-back, ambient type melodies. The result is still a bit too random for my tastes, but I think this has potential. So, listen here:

http://www.box.net/shared/r6j1jtla80

I went for a more peaceful ambiance this time around. Also, shamisen FTW.

EDIT: Also, the first post now has a link to a flash widget with all the mp3s.
« Last Edit: July 13, 2008, 06:25:00 AM by muku » Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
muku
Level 10
*****



View Profile WWW
« Reply #6 on: July 13, 2008, 08:47:43 AM »

Heh. How 'bout some good old-fashioned true procedural metal? Perfect for those blood-thumping boss fights, I reckon. See Flash link in first post to listen.

It has much the same problems as the last one, being just a bit too random, plus the solo notes sometimes get a bit too high. But that's okay, at the moment I'm just sketching out different ways in which this could go, I can finetune later.

EDIT: Bonus points for anyone who can identify in which modes the musical snippets so far were Grin
« Last Edit: July 13, 2008, 08:57:10 AM by muku » Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
Lord Ash
Level 0
**



View Profile
« Reply #7 on: July 14, 2008, 03:42:18 AM »

muku, that is awesome !!

and all that just from some maths huh, way cool ! Gentleman
Logged
PaulMorel
Level 1
*



View Profile WWW
« Reply #8 on: July 14, 2008, 06:12:04 AM »

Good stuff, ash!  I have always wanted to implement "real" procedural music (as opposed to the pseudo-procedural music in my PGC entry, Last Colonists), but I have never really had the time.

There is a nice piece of professional software devoted to procedural music that I thought you might like.  It's called Noatikl, and it can be found here: http://www.intermorphic.com/tools/noatikl/

I have been very close to buying it for about a year, but I have never been able to really justify it as anything other than a toy.

I also wanted to note that there are a number of algorithms for generative music that you might find interesting.  If you already have the framework set up, then implementing the algorithms shouldn't be too tough.  I wish wikipedia had better entries on these...
http://en.wikipedia.org/wiki/Algorithmic_composition
http://en.wikipedia.org/wiki/Generative_music

Specifically, Chomsky's linguistic models have been used very effectively.  Those models give the computer a knowledge of music syntax.  The computer knows what it is doing now and what it has done in the past, so it can make an educated guess about what to do next based on a sort of musical grammar.

And also the mathematical models have been used to great effect.  As opposed to Chomsky's style of model, mathematical approaches rely less on an understanding of the rules of music (a dominant harmony usually follows a subdominant), and more on numbers-based, probabalistic approaches.

Obviously, that's a vast oversimplification, but hopefully that's useful to someone...
Logged

my computer music blog: http://www.thisisnotalabel.com
muku
Level 10
*****



View Profile WWW
« Reply #9 on: July 14, 2008, 06:29:45 AM »

Good stuff, ash!
You got the names mixed up  Wink

Quote
There is a nice piece of professional software devoted to procedural music that I thought you might like.  It's called Noatikl, and it can be found here: http://www.intermorphic.com/tools/noatikl/
Might have to look at it, perhaps there's a demo...

Quote
I also wanted to note that there are a number of algorithms for generative music that you might find interesting.
Definitely. So far, I just went off my intuition for algorithms, but it would certainly be nice to benefit from the research others have done in that area. You wouldn't happen to have any links to papers which describe relevant algorithms? I have just started scouring Citeseer, that might turn up some interesting stuff.

On the other hand, I want to keep computational effort down so that generating music in realtime in an actual game would still be feasible, but it certainly can't hurt to look at the state of the art in algorithmic music.

One "problem" is that much of the generative music community seems to be concerned with generating rather abstract soundscapes instead of more traditional interpretations of music. That's of course a valid artistic choice, and it certainly makes sense to go for new directions which only algorithmic music generation has made possible, but I here am going for more conventional stuff that you might find in your average indie game soundtrack.
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
Alex May
...is probably drunk right now.
Level 10
*


hen hao wan


View Profile WWW Email
« Reply #10 on: July 14, 2008, 07:18:30 AM »

I'm extremely impressed with this. Good work! Look forward to seeing how it comes along.
Logged

muku
Level 10
*****



View Profile WWW
« Reply #11 on: July 14, 2008, 07:36:15 AM »

Thanks for all the encouragement, I appreciate it. I've just skimmed an interesting paper off Citeseer which described an approach to create jazz solos via a genetic algorithm. I'm itching to put some of those ideas into code (AI buzzword alarm notwithstanding), though I'm a bit wary about performance issues. But we'll see.

For now, though, I have to put some hours into real-life stuff. Curse you, diploma thesis! *shakes fist*
Logged

The Cosyne Synthesis Engine - realtime music synthesis for games
J
Level 0
**


'Gotta roll with it...


View Profile
« Reply #12 on: July 14, 2008, 08:58:26 AM »

Heh. How 'bout some good old-fashioned true procedural metal? Perfect for those blood-thumping boss fights, I reckon. See Flash link in first post to listen.

It has much the same problems as the last one, being just a bit too random, plus the solo notes sometimes get a bit too high.

Despite that one instance of insanely high tones compared with the rest of the piece, i'm still astonished that these are procedurally generated, very well done, loved the metal.
Logged

PaulMorel
Level 1
*



View Profile WWW
« Reply #13 on: July 14, 2008, 10:27:54 AM »

Sorry for mixing up the names.  To be more correct: Good stuff, Muku!

On the other hand, I want to keep computational effort down so that generating music in realtime in an actual game would still be feasible, but it certainly can't hurt to look at the state of the art in algorithmic music.

One "problem" is that much of the generative music community seems to be concerned with generating rather abstract soundscapes instead of more traditional interpretations of music. That's of course a valid artistic choice, and it certainly makes sense to go for new directions which only algorithmic music generation has made possible, but I here am going for more conventional stuff that you might find in your average indie game soundtrack.

The first problem there is definitely a problem.  Computational complexity can get quite high, and when you need to spend most of your time painting the graphics, it's tough to devote any extra time to the music.  This is compounded when you consider the memory required for realistic sampling.  If you want patches that sound better than general midi, then you are stuck devoting tons of memory to audio files.  The extra CPU and memory use is why no mainstream games really use generative music.  AFAIK Spore will be the first to do truly generative music, and even then, I bet that they simplify it in a bunch of ways (using synths instead of sampled instruments, simplifying the math to make it quicker .. etc).

That second problem is only illusory.  There is a TON of research on generative music and the algorithms to do it.  I wish I had some of them handy to show you.   Sad

One relatively new music model that I absolutely LOVE is being peddled by a guy named Dmitri Tymoczko.  His model is amazing, not only in its completeness, but also in its complexity.  It has massive implications for generative music, but is probably far beyond the scope of your project.  I wrote about it on my blog last week: http://www.thisisnotalabel.com/talk.php?bp=524
Logged

my computer music blog: http://www.thisisnotalabel.com
Cymon
Level 9
****


Computer Kid


View Profile WWW Email
« Reply #14 on: July 14, 2008, 09:04:22 PM »

Okay, my 2 cents. When I was younger I played a game called Star Control and I thought it would be awesome if when I got older and could write a new Star Control if the melee music would shift depending on who was winning. I devised a method where by two songs that each had similar hits could be faded back and forth. Then I thought that would be boring and fiddled with something like this, where the music would actually be generated by the variables of the game. This would be perfect for that since in Star Control Melee the music really just sets the mood.

Also, didn't Douglas Adams write about something like this in "Dirk Gently's Holistic Detective Agency"?

Now cool as it is that you have music that was created san human hands, it's not very good music. It's still kinda random and it lack a "hook". However, can these things be added without destroying (or limiting) the procedural nature of the output? How would using variables decided at the beginning to set a beat box and keep it going the whole time to give it continuity?
Logged

Cymon's Games, free source code, tutorials, and a new game every week!
Follow me on twitter
Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic