Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411276 Posts in 69323 Topics- by 58380 Members - Latest Member: bob1029

March 28, 2024, 12:05:57 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperArt (Moderator: JWK5)Vector Graphics Rendering with Unity and Audio (a work in progress)
Pages: 1 [2]
Print
Author Topic: Vector Graphics Rendering with Unity and Audio (a work in progress)  (Read 23874 times)
ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #20 on: April 02, 2016, 01:57:40 AM »

Logged

Pishtaco
Level 10
*****


View Profile WWW
« Reply #21 on: April 02, 2016, 01:47:22 PM »

That's neat. I like the way it animates.

Inspired by this thread I've been working on an oscilloscope simulator (using some code from http://m1el.github.io/woscope-how/index.html ).

It's looking like it isn't easy to get audio output from one program and pipe it, in real time, into another. There are some "virtual audio cable" tools floating around, that simulate an audio output device at one end and an input device at the other, but they seem tricky to use and I haven't got one working in stereo. Have you had any luck doing something like this?

(And unfortunately I decided to do it in html5, and some browsers don't give you access to audio input.)
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #22 on: April 02, 2016, 08:28:34 PM »

That's neat. I like the way it animates.

Inspired by this thread I've been working on an oscilloscope simulator (using some code from http://m1el.github.io/woscope-how/index.html ).

It's looking like it isn't easy to get audio output from one program and pipe it, in real time, into another. There are some "virtual audio cable" tools floating around, that simulate an audio output device at one end and an input device at the other, but they seem tricky to use and I haven't got one working in stereo. Have you had any luck doing something like this?

(And unfortunately I decided to do it in html5, and some browsers don't give you access to audio input.)

That seems to work really well!

Currently I have a system that can read up to 8192 samples per frame in Unity directly from the Audio Listener, and it works really well, though I'm inputting that data directly into the built in script Line Renderer, and results are... ehhh. I'd like to do something better but I'm not too experienced with programming visual things... yet.





Basically to figure out how many points to show at any given moment, I take the total samplerate per second and divide it by frames per second, and it works out perfectly. I just need to figure out a better way to render the data cause this line renderer is garbage for this stuff, but at least it works.
Logged

Pishtaco
Level 10
*****


View Profile WWW
« Reply #23 on: April 10, 2016, 02:22:10 AM »

Getting somewhere:


That's running in chrome. I managed to dig up the undocumented command that let's chrome get stereo mic input - it should also be in the next version of firefox.

Current-ish version here: https://dl.dropboxusercontent.com/u/29654821/oscilloscope.html

I don't get what "r+click" does in electricanvas. If I hold the r key, and left click, I get a line going off screen; but maybe the oscilloscope version is supposed to be a disconnected point?

Something goes wrong, I guess with my low pass filter, when I set accuracy to 0.

I'm using this http://vb-audio.pagesperso-orange.fr/Cable/ in windows to pipe default audio output back to something pretending to be a microphone, so it's important to turn the audio volume down in the oscilloscope to avoid feedback.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #24 on: April 10, 2016, 02:35:18 AM »

Here's another demo of the current scope simulation. My computer was being a little problematic while recording earlier but it's generally rather stable most of the time.


It looks great at 60fps and it's rather close to the same image as on the real scope!




I'm hoping to roll out the new version of Electricanvas, 2.0, sometime in the next week. The whole thing is done and working, I'm just trying to put together a cleaner more responsive GUI, though the only thing in that is the interpolation rate and instructions. Other than that, it's completely ready. Here's some of the enw features.

- uses the new VectorWorks script and system which is much more stable and featured
- hold E to drag current point, or T to drag the origin point
- dragging is smoothly animated with the new smoothing algorithm
- holding down R while clicking (or dragging with E or T) produces a disconnected point for disconnected geometry
- the whole display is now an accurate simulation of the audio you draw rather than just reading the points from memory like the old version. You no longer need a scope to see what it'll look like!

I might add functionality to mess with audio effects if I can get the GUI functional enough to do that, since it's really incredible to watch and manipulate your drawings with some of the effects.


EDIT: oh wow response while I was typing, pff.

@Pishtaco Yeah, the line goes off screen but only in the preview, it's more a problem of how i coded it since I needed a way to mark the coords that were being disconnected so the renderer would do it and at some point I gave up trying to fix the preview part. It was before I used Vector3s for everything so instead I just had it add 100 units to those lines and the audio renderer would pick up on what was over 80 and subtract a hundred as well as skip interpolation for those lines. It was a mess, but the new version should make things much nicer.

EDIT EDIT: Oh wow that works and looks really really good! Damn! The only big issue being that I guess the method it gets it's audio results in a lot of jitter and junk, but wow, it really works well! All the scope artifacts look gosh darn beautiful.
« Last Edit: April 10, 2016, 02:45:38 AM by ompuco » Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #25 on: April 18, 2016, 11:30:12 AM »





So geometry culling is turning out to be somewhat of a challenge, at least for me. More info in the video description, but I may need to seek out help sooner or later, at least with figuring out how to get some functions to work.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #26 on: April 19, 2016, 07:29:54 PM »

Logged

Pishtaco
Level 10
*****


View Profile WWW
« Reply #27 on: April 28, 2016, 02:06:40 PM »



My oscilloscope sim is nearly done. If anyone wants to try it: https://dl.dropboxusercontent.com/u/29654821/oscilloscope%20demo/oscilloscope.html
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #28 on: May 26, 2016, 01:13:44 PM »

been working a lot on this lately. i rewrote the whole rendering system and allowed for distance-based interpolation so that the beam is always moving at the same speed and the resulting image is a lot sharper while bringing up the refresh rate significantly. also been messing with strokes and such to see about making it good enough for use for actual drawing.





once i finish up work with the new version of electricanvas, which involves proper scope simulation and a vastly improved GUI, I'll be getting back to the 3D work since the rendering system has gotten so optimized.

also been studying shaders a lot lately and really want to find a way to implement it with the renderer better, possibly rendering the audio within the GPU/shader itself so i can access absurdly complex mesh data and sweep through it without slowdown. I've seen WebGL examples that do audio rendering and it works roughly the same, but I still have no idea if shaderlab shaders can even write to audio channels in Unity, and I haven't seen any examples that do it.
« Last Edit: May 26, 2016, 01:26:16 PM by ompuco » Logged

readyplaygames
Level 2
**


View Profile WWW
« Reply #29 on: May 29, 2016, 12:03:54 PM »

This is really, really cool. The gifs alone are awesome.
Logged
ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #30 on: November 05, 2016, 11:20:37 AM »

Project went on hiatus because of a move and losing the connector for my scope in that move. Finally got a new connector made about a week or so ago after picking up a new serial connector and soldering iron that wasn't total ass. I did this as a refresher while revamping the audio renderer with recent things I learned.



I've also been experimenting with a new, highly optimized scope emulator in Unity, though it's not nearly as accurate as Pishtaco's so I'm hoping to eventually find a way to or get help with recreating it entirely in Unity, since I keep finding myself getting lost when trying to dissect the HTML/Javascript code.

Currently looking into very very low level graphics programming and am considering if I should just learn plain C# and how to make a 3D OpenGL program in VS, since I'm needing to access not just GPU vertex coordinates, but after vertices have been culled and clipped. Talk about Unity's experimental Scriptable Render Loops makes me slightly hopeful that, should the feature ship, I might be able to get this GPU data while still using just Unity, which would be great, but I'm not sure entirely what it entices.

Hoping to have more to share very soon, including more programs and very possibly a script/asset.
Logged

muki
Level 2
**



View Profile
« Reply #31 on: March 17, 2017, 01:12:44 PM »

I am in complete awe on how amazing everything in this thread looks!

One of my other hobbies is electronics (I design/build guitar pedals) and also play around with scopes. I've played around with DACs and micros outputting fractal-ish stuff to the scope, but nothing nearly as amazing as all of this!
Logged
Pishtaco
Level 10
*****


View Profile WWW
« Reply #32 on: April 20, 2017, 01:32:13 PM »

My webgl oscilloscope emulator is done: https://dood.al/oscilloscope/

Logged

leolodreamland
TIGBaby
*


View Profile
« Reply #33 on: April 20, 2017, 04:56:27 PM »

i would love to see some assets. i've just been playing with my scope that someone gave me and it's right up my street...
Logged
Pages: 1 [2]
Print
Jump to:  

Theme orange-lt created by panic