Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411273 Posts in 69323 Topics- by 58380 Members - Latest Member: bob1029

March 28, 2024, 03:05:01 AM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Rhythm game audio/visual synchronization?
Pages: [1]
Print
Author Topic: Rhythm game audio/visual synchronization?  (Read 5025 times)
BenH
Level 2
**


Dirty boy, wash your hands!


View Profile WWW
« on: January 21, 2008, 07:18:04 AM »

Hey guys

I've always wanted to try making a music rhythm game like say Parappa, Vib Ribbon or Rhythm Tengoku, hell even Guitar hero I guess. Anyway, the problem I've always had trouble getting my head around is how to keep the audio and the visuals/code synced up so the game plays correctly. Audio is nearly always played on its own thread, so if the code/graphics slow down, the audio carries on quite happily. Obviously that would cause a big problem when having to check for correct beat 'hits' in the code.

Any help would be much appreciated! Smiley
Logged

ravuya
Level 7
**


Yip yip yip yip yip


View Profile WWW
« Reply #1 on: January 21, 2008, 08:07:00 AM »

Not quite sure how I'd do this exactly, but I'd assume your API will let you apply some kind of callback function to fire periodic "timing messages" back to the main thread, like a metronome.

Every n seconds or so of the audio thread, you could tell the main thread to synch back up. Not quite sure how you'd do that, but it should work out fairly well.
Logged

BenH
Level 2
**


Dirty boy, wash your hands!


View Profile WWW
« Reply #2 on: January 21, 2008, 11:24:18 AM »

I'm not 100% sure what I'd actually be making this in, so I can't really rely on higher end audio stuff like call-backs, but I thought perhaps using something like the current song position could work. I guess I'll have to try it out!

If anyone else has any input, that'd be awesome. Smiley I've done some searching around on the net and I can't seem to find anything related to this topic, but I might be looking for the wrong things.
Logged

Corpus
Guest
« Reply #3 on: January 29, 2008, 01:12:08 PM »

You could always quickly pause and unpause the music track in time with each tick, impulse or whatever you want to call it of the game code's actual update script. That way, if it was being played without slowdown, the music would run smoothly, and people experiencing slowdown would know straight away that it's their computer's fault, and not yours, because the music would be juddery, too.

The idea of synching stages in the music to stages in the code (which could be achieved quite easily (theoretically, at least) by dividing the song's time into a number of sections and then resetting the music back to one of those 'waypoints' if it becomes too far unsynched) may be more effective and produce a smoother-sounding output, though. It's hard to tell, without actually testing the method.
Logged
queasy
Quite Possibly Jon Mak
Level 0
*


View Profile
« Reply #4 on: January 29, 2008, 01:45:46 PM »

hey, it might not be as complicated as you think it is.  everyday shooter did a lot of game logic synching to music, sometimes even towards the end of the level which was minutes away.

the main thing i did was just use the song position as the time to sync too.  So instead of saying "spawn x guys in 30 seconds", i say "when the song time is at 1:31, spawn the guys."

Assuming, your music tempo/bpm is rock solid, you can also use the current song pos time to calculate how close you are on the beat etc.  In ES it was a little more difficult because I suck so much I can't play on the beat.

Anyway, hope that's helpful.

Oh another thing is, for things where you can't really synch it to the song pos all the time, just make sure that your game engine will always catchup if it loses time.  IE. skip frames until your simulation is in time with the music again.

-j
Logged
BenH
Level 2
**


Dirty boy, wash your hands!


View Profile WWW
« Reply #5 on: January 29, 2008, 03:29:44 PM »

Thanks for the help, Jon. I knew I was making it overly complicated for myself!
Logged

soundofjw
Level 2
**



View Profile WWW
« Reply #6 on: February 27, 2008, 08:55:03 AM »

Hey.
I recently created a sound extension for MMF2 and the biggest hurdle I faced was figuring how to get certain events to trigger at a given time despite the latency that might be faced from the sound buffer.

Essentially when you stream or playback audio, you fill the sound buffer with audio. You can imagine, however, when your sound buffer is two seconds long, that you write audio data that is two seconds in advance from when you actually write the data.

To overcome the audio/visualization sync. problem I had was to create a "VizSync" system.

Essentially, you need to store some sort of table of data (I used a link list, method) for every potential sample in the audio buffer. Everytime you write you'll note the time in the 'song' of the information your writing (FFT data, sample data, whatever you want... module row or order information, anything). Then, when you need to retrieve the latest data, you pass the current playback time of the buffer to a method that traverses the linked list or table from the last entry that it retrieved until it finds the entry with the latest playback time that isn't greater than the current time that you've passed.

Gosh I hope that made ANY sense.
Logged

hi
siiseli
Level 6
*



View Profile
« Reply #7 on: February 28, 2008, 07:20:44 AM »

If you just take time with a timer to see how long it takes to do one frame, and use that to "change" some of the variables in the game(those that have to do with moving and stuff). Like..

while(key[KEY_ESC]){
start_timer();

x+=dx*dt
if(key[KEY_RIGHT]) dx+=100*dt
if(key[KEY_LEFT]) dx-=100*dt

dt=get_timer();
}

You propably get what I mean, at least I hope so.
Logged
Flink
Level 1
*


Visiontrick


View Profile WWW
« Reply #8 on: February 29, 2008, 05:54:05 AM »

I once did a small 2D demo with everything synchronized to the music. I used FMOD fort it and it was really easy cuz you could get like the "volume" from the songs different audio channels and then let the demo depend on that.
Logged

MattJ
Level 0
***



View Profile WWW
« Reply #9 on: March 11, 2008, 05:07:24 AM »

For OokiBloks we have a system based on the current song play position, basically what Jon is suggesting above. We have a custom tool to place sync points in the audio manually, the tool automates adding loads of regular beat points but also allows fine tuning and adjustment. On playback when the music passes a sync point (allowing for buffering, which is a fixed adjustment) an interrupt is fired. In the sync interrupt you could handle any requests you've had for an on beat sound effect, or you can make a request for some on beat gfx to fire.

In Ooki the tool has allowed us to add loads of other information about the musical key and how groups of sounds associate with each sync point. Which in turn allowed us to trigger sounds more intelligently in the interrupts.
Logged

scruffy hermitgamestwitter
Golds
Loves Juno
Level 10
*


Juno sucks


View Profile WWW
« Reply #10 on: March 11, 2008, 05:59:20 AM »

With FMOD, you can use FMOD_Channel_GetPosition() to get the current playback time on a channel in milliseconds.  you can poll that, and work from there.

Assuming you have a song in a constant time signature at a constant tempo, you can manually keep some data on it like beats per minute, time signature (how many beats per measure), etc, and when your channel position is close enough to events that you want to trigger on or around these beats, then do something special, etc.

I think FMOD also does FFT/spectrum analyzation as well, so you can do stuff not just based on currrent volume, but relative intensity at various pitches.  i haven't played around with this though.
« Last Edit: March 11, 2008, 06:06:55 AM by Golds » Logged

@doomlaser, mark johns
Pages: [1]
Print
Jump to:  

Theme orange-lt created by panic