Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1026285 Posts in 41135 Topics- by 32738 Members - Latest Member: Yarif

July 24, 2014, 08:54:26 AM
TIGSource ForumsFeedbackDevLogsFrequency Domain [ Now with Play Create Share. Play the build! ]
Pages: [1] 2 3 ... 5
Print
Author Topic: Frequency Domain [ Now with Play Create Share. Play the build! ]  (Read 7387 times)
sagzorz
Level 0
***


View Profile WWW
« on: February 04, 2013, 06:46:50 PM »

Frequency Domain



You can now use your own mp3 files and edit the track in game!
Export your parameters and share!

Download Builds Here (v0.5, also Oculus Rift builds):
http://steamcommunity.com/sharedfiles/filedetails/updates/142393387/1372116927



Feedback is appreciated!


Status: Active Development
Dev Team: Solo
Platforms: PC,  Mac
Framework: Unity 3D

Steam Greenlight Concept page: http://steamcommunity.com/sharedfiles/filedetails/?id=142393387
General Devlog: http://sagzorz.com
Source Code: https://github.com/SagarPatel/FrequencyDomain
IndieDB Page: http://www.indiedb.com/games/frequency-domain




Some Background

Back in May of 2012 I participated in TOJam 7 with 2 friends (sound designers) John Axon and Marius Masalar. For those who don’t know, TOJam is a 48 hour game jam that takes place yearly in Toronto. I HIGHLY recommend going to a TOJam, they’re incredibly well organized and it’s always a great experience.

Going into May 2012′s TOJam, we knew that we wanted to make a game that used audio as a core part of the gameplay. Considering that we were 2 audio guys and 1 programmer (me), it seemed like the best used of our skills, although we weren’t sure yet exactly what to do with the game.

I arrived in Toronto the night before the jam started and at this point in time I had dabbled with Unity a little, but wasn’t very comfortable with it. So I decided to poke around the documentation and eventually stumbled upon the “AudioSource.GetSpectrumData” function. This function essentially performed Fast Fourier Transform (FFT) in real time on the audio/music being played and returned the frequency domain data at a given resolution.

To keep it short, the FFT data was used to to move platforms (placed in rows) up and down and the FFT data was also used to control the color of said platforms. We ended up creating a first person puzzle-platformer, where the player could manipulate music, which in turn would affect the platforms directly and allow the player to traverse the environment. We gave the player control over what musical tracks to toggle on or off as well as volume control, all of which would directly contribute to the movement of the platforms, allowing the player to reach new areas of the environment if used correctly.



There is much to be said about the TOJam game and the various design and technical dilemmas we encountered and why I halted further development of it, but I’ll save that for the dedicated TOJam 2012 project page.


Onto Frequency Domain

The Frequency Domain concept came from my desire to streamline the design of the TOJam 2012 game to it’s core and to remove the awkwardness of the first person platforming experience. I planned to improve on the main tech from the jam game, with the focus being on creating a environment that would be continuously shaped and colored in real time via the FFT data.

Unlike the TOJam game, I wanted to have a gap-less continuous surface for the player to travel on, as to avoid the awkwardness of first person platforming. By this I specifically mean moments when the player is unsure if they’ll manage the jump and has to resort to regularly looking down to know where the edge of the platform is. That specific issue occurred rather often in the TOJam game and understandably so, the mechanics of it made somewhat of a “precision platformer” (more on that and other issues on the TOJam 2012 page).

The obvious solution was to use a large mesh where the height of the vertices would be attached to the FFT data: no gaps, no awkward platforming moments. In the TOJam game, all the platforms reacted in real time to the music, and since the swings of amplitude in FFT data are often violent, having a surface dictated solely by real time data (raw, unprocessed) makes for a rather difficult, if not unplayable, experience. That’s when the idea of having a “2D field mesh” defined by 2 axes came to me: one axis would be the frequency values and the other would represent time. The frequency patter (i.e. FFT data) would start at one end of the field and  would make its way down the mesh, 1 row at a time every update loop.

Here’s some of the first footage of the mesh in action:
http://www.youtube.com/embed/E-PZMsENml4


The video above is with a single track audio. It’s maxing out Unity’s FFT function to fetch 8192 points of data, of which only the first 128 are used in this case. So what we’re seeing in the video is are the low frequency sounds, mainly beat and bass guitar in this case of this song.

The end goal is that the player can explore/navigate this terrain as its being generated, think first person Sonic. Having multi-track audio data will allow me to create a mesh for each track – players gets to explore and switch between the different tracks on the fly and have the meshes be surrounding them in a ring like formation.

....much more to come (just got to write it up)...
« Last Edit: June 25, 2013, 08:17:08 PM by sagzorz » Logged

aliceffekt
Level 4
****





View Profile WWW Email
« Reply #1 on: February 06, 2013, 04:47:10 AM »

Looking good : ) 

Can't wait to see more. I saw a cool vine animation you posted today too.  Beer!
Logged

XXIIVV

Drypoint, Cyanosis, Kanikule, Valentinel, Pierrot Death, Zjeveni, Merveilles, Cenote, Cloud is a Lie, Pico³, Siseon°, Phloston Paradise, Volkenessen, FRACT, Waiting for Horus, Diluvium, Pico Battle, Hiversaires, Melady Antres.
Thecoolestnerdguy
Level 2
**

Can't code without autocomplete.


View Profile
« Reply #2 on: February 06, 2013, 05:44:57 AM »

Looking good : ) 

Actually, sounding good. Cheesy got it? got it?
Logged

Sad minecraft didnt copyright squares
Number
Level 0
**


View Profile
« Reply #3 on: February 08, 2013, 07:36:08 AM »

Great demo, what's the music?
Logged
sagzorz
Level 0
***


View Profile WWW
« Reply #4 on: February 10, 2013, 07:42:39 AM »

Looking good : ) 

Can't wait to see more. I saw a cool vine animation you posted today too.  Beer!

Thanks!
And here's the gif (it's a plain old gif, not Vine I'm afraid, can't wait till it gets on Android):

Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #5 on: February 10, 2013, 07:47:07 AM »

Great demo, what's the music?

The music is from The Electric Bends, specifically "Unknown Intent" from their latest album, Plastic Fantastic (in which I guest start as a Tabla player  Cheesy )

electricbends.bandcamp.com
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #6 on: February 10, 2013, 01:59:48 PM »

The previous video is straight up raw data, that's why the peaks are so pointy and narrow: not the best surface to try to ride on. It's part of the reason why I wanted to get a "blockier" look and feel to the surface. The way I'm doing this right now is to simply repeat data points several times.
Here's what that looks like:

http://www.youtube.com/watch?v=HQpmrrn_a-Q
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #7 on: February 20, 2013, 01:57:50 PM »

Tech Test 4

Official post: http://sagzorz.com/2013/02/11/frequency-domain-tech-test-4-push-it-to-the-limit/

Implemented the most rudimentary heightmap based physics where the closest vertices corresponds to the player height (barely took a minute or two). The next step is to use the 4 closest points to the player (i.e. the “square” its in) and calculate the precise height from it.

On the graphics side, I’m now experimenting with Unity’s Parallax Diffuse shader, I’m also using 2 strong point light sources at the front to create the intense white light fromm which the notes “emerge”.



It should be mentioned that the audio data being used in the Tech Test 4 video is still raw/unprocessed: the only thing I’m doing at the moment is selecting an optimal FFT data resolution and a frequency range to go with it. I was rather surprised by how well the FFT picked up the vocals in this song (make sure not to miss the end, it gets pretty epic). I had tried some Barry White previously but the vocals didn't get picked up very clearly, maybe because his deep/low voice got drowned out by the instrumentals. In any case, I’ll need to test more vocal heavy tracks and see what kind of results I get, this just might get really interesting.

Push It To The Limit --> http://www.youtube.com/watch?feature=player_embedded&v=4Rf1-BhWaEc
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #8 on: February 27, 2013, 02:56:29 PM »

Seriously tightened up the graphics on level 3.
Restored the visual feel of Frequency Domain post tech upgrades.
On the verge of awesomeness.

Will have massive post soon.
Maybe a tech gif tonight to get started.
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #9 on: February 28, 2013, 12:57:24 AM »

Frequency Domain: Tech Test 5 – Heightmap Lerp and Pseudo Log Scale

Official Blog Post: http://sagzorz.com/2013/02/28/frequency-domain-tech-test-5-heightmap-lerp-and-pseudo-log-scale/

Completed 2 of the biggest technical things on the to-do list earlier this week:

- Getting a height reading that corresponds perfectly to the terrain (even between vertices)

- Implementing a flexible pseudo log scaling solution for the FFT data. This would allow me to manage the entire audio spectrum and have it all on the field (if need be). Previously I was only using a small portion of the FFT data at medium resolution, which resulted in missing out on some of the frequencies.

Here’s a gif demonstrating the heightmap inter-vertices lerping I’m doing to get a good match with the terrain.



Next is laying out some static environment to help the player better understand his/her movements in the world. I’m going to try to sneak in the boost rings in as well (more on that later).

Once that’s in, implementing some form of velocity based movement for the player will be key. This will allow me to properly test the “wave riding” mechanic I had in mind. This is where is should start to get “fun” to play.
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #10 on: February 28, 2013, 10:38:16 PM »

Gave a little tech talk today at Concordia about Frequency Domain.
Got a post up with the major technical points I covered:
http://sagzorz.com/2013/02/28/a-technical-breakdown-of-frequency-domain/
Logged

DustyDrake
Level 10
*****



View Profile
« Reply #11 on: March 03, 2013, 04:30:23 PM »

The problem that I see with this game is that not a lot of songs offer a good area to drive through or explore.
Like for example you've usually got this large mountainous area to the left, and then seemingly chaotic bumps in the middle, and then barely anything interesting to the right.

Personally, I think the platformer sounds like a cooler idea.
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #12 on: March 03, 2013, 05:06:14 PM »

The problem that I see with this game is that not a lot of songs offer a good area to drive through or explore.
Like for example you've usually got this large mountainous area to the left, and then seemingly chaotic bumps in the middle, and then barely anything interesting to the right.

Personally, I think the platformer sounds like a cooler idea.

Yeah, the frequency spread was an issue as you pointed out (too much on left, not enough on right). This was mainly due to the fact that I was using the FFT data as Unity gave it to me, i.e. on linear scale. Music doesn't really work that way, it's easier to see on a logarithmic scale.

I recently got around to implementing a pseudo-log scale for the data that's flexible, allowing my to tweak the data on the fly (blog post about it here --> http://sagzorz.com/2013/02/28/a-technical-breakdown-of-frequency-domain/

This results in a far more balanced spread in the terrain.

Here's a gif of the current build (give it a moment, it's rather large).



I'll have a video up asap, it'll make things much more clear  Grin

As for the original platformer idea, it's something I'd like to go back to someday. There are really good reasons why I put the project on hold and started Frequency Domain, I'm going to have to write that post-mortem on of these days >___<
Logged

sagzorz
Level 0
***


View Profile WWW
« Reply #13 on: March 03, 2013, 09:13:18 PM »

Playable Prototype 1 (ft. @aliceffekt ) --> http://www.youtube.com/watch?v=eUddGEzdJyg&feature=youtu.be

I assure you it looks better when playing it  Grin
Logged

Raycast
Level 1
*


MAGDEV


View Profile Email
« Reply #14 on: March 03, 2013, 09:50:32 PM »

Dang, that looks pretty. Looking forward to seeing how this turns out.
Logged

Pages: [1] 2 3 ... 5
Print
Jump to:  

Theme orange-lt created by panic