Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411486 Posts in 69371 Topics- by 58427 Members - Latest Member: shelton786

April 24, 2024, 09:03:37 AM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperArt (Moderator: JWK5)Vector Graphics Rendering with Unity and Audio (a work in progress)
Pages: [1] 2
Print
Author Topic: Vector Graphics Rendering with Unity and Audio (a work in progress)  (Read 24261 times)
ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« on: February 25, 2016, 11:41:09 PM »


so over the past month i finally started working on my own vector graphics rendering system that outputs XY data through stereo channels on the DAC which is hooked up to an oscilloscope or arcade vector display (or in my case, the XY testing pins on an old Tektronix 1720 Vectorscope).

for about a year or two i've really wanted to see a game made using these traditional displays since for a while i saw a ton of games trying to emulate the look without really understanding the technicalities that make it so interesting, and also hot damn these displays are really impressive and ridiculously fluid, as well as looking slick as hell. also, while there were a lot of people making games that looked like they were on a vector monitor, it was pretty lame seeing as there wasn't really anything at all being made that could actually be played on one, which was pretty upsetting.


currently i've gotten a basic version of the rendering system done within unity, and released a bit of a demo program for drawing onto a vector display using audio. currently it's kinda buggy and bare bones, but i've been adding a little to it and hope to update it soon.





you can download electricanvas here. it's a lot of fun if you're using it with a scope, though.


the next thing i want to do is figure out a way to read geometry of objects in the unity scene so that i can get started with the actual project i had in mind since the beginning of this endeavor, which is a 3D exploration game designed entirely for an analog vector display.



i'll be updating this thread as i get further through work on this system and this project in general. wasn't sure if it would be appropriate in devlogs or another thread since this is still technically too early to say it's a game, and while it is a technical/programming challenge, i'm focusing a lot more on the aesthetic results i'm getting through this and wanted to share this with people more oriented in art. if a mod thinks this would be much more at home on another board here, than feel free to do so, but for now i feel like it's at least sorta okay under Art.
« Last Edit: February 25, 2016, 11:46:11 PM by ompuco » Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #1 on: February 26, 2016, 05:26:41 PM »



Electricanvas was updated to version 1.02 which includes better rendering and the ability to draw disconnected geometry.



I'll be working on rewriting everything for the 3D engine since it's a bit of a mess, but now I've assembled a pretty comfortable playground for working with these graphics. Got a lot of time this weekend so I'm hoping to make decent progress with the 3D renderer.


Logged

RyanB
Level 0
***


FIRE + SKULLS = AWESOME


View Profile WWW
« Reply #2 on: February 27, 2016, 02:53:14 AM »

Dude!  You rock!

I went to school for electronics and I made an animation that played through my oscilloscope for a class.  That was just shifting data around, this is the next level and beyond.   Wizard

Most people won't get it but this is super cool.  I wish I had time to hook it up to my oscilloscope (it's analog!).

Logged

diegzumillo
Level 10
*****


This avatar is so old I still have a some hair


View Profile WWW
« Reply #3 on: February 27, 2016, 08:48:24 PM »

Finally you made a topic for this project! I've been waiting for it, this looks awesome. If you ever make something public I may be able to find a lab in my college with an oscilloscope I can borrow. If not, I maybe able to find a lab in my college with an oscilloscope I can sneak out.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #4 on: February 27, 2016, 11:40:10 PM »

Ended up being kinda busy over the past two days but I'm gonna try rewriting the system now since the old scripts are a serious mess, and now at least I actually know how to get things going without writing up hundreds of lines for experimentation.

Still need to figure out how to get vertex coordinates to send in relation to the camera. I tried to do it with mesh.vertices and WorldToScreenPoint but the vertices code only gets local geometry data instead of something relevant to the camera, resulting in it only showing a poor representation of an object's static geometry. I might try looking into if I can use custom shaders to feed GPU data to the vector renderer, but it might be a lot more complicated than I first thought.

Though if anyone can give some tips on how to do this, I'd love to hear!




Finally you made a topic for this project! I've been waiting for it, this looks awesome. If you ever make something public I may be able to find a lab in my college with an oscilloscope I can borrow. If not, I maybe able to find a lab in my college with an oscilloscope I can sneak out.

Electricanvas is currently publicly available! Wanted to get that program out there so that people can test it out so i could see how well the code works with other computer/DAC setups.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #5 on: February 28, 2016, 10:54:12 PM »


Still having a ton of difficulty, mainly trying to use transform.TransformPoint across a list of Vector3 coords. Spent like three hours trying to figure that out and gave up and just decided to type a line for each and every vertex, which was NOT fun.

I'm sure there might be a better way to get vertex coordinates in relation to the world (or better, the camera), but this is the first time I actually got something to work right.

At least I got results at long last, so that's cool!
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #6 on: March 04, 2016, 04:43:30 AM »



Logged

kuubaas
Level 0
**



View Profile
« Reply #7 on: March 08, 2016, 02:08:41 AM »

Wow.

Games for oscilloscopes is a very underexplored market niche.
Logged

technical artist for no truce with the furies
Tanner Fruit Fly
Level 0
***


Tanner sticks the pen up his nose.


View Profile WWW
« Reply #8 on: March 08, 2016, 03:18:28 PM »

Wow, this premise is so stunning and just visually intriguing. I was mesmerized by that last video you uploaded, procedural art seems like such a crazy thing to attempt and I think you've nailed it here. Really fun to watch Smiley
Logged


[Check out the "This Is Jim" book on Gumroad! Head over to my devlog for a discounted promo code Wink]
ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #9 on: March 11, 2016, 09:29:23 PM »


Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #10 on: March 15, 2016, 03:02:57 AM »



Logged

Mannapi
Level 0
***


.


View Profile
« Reply #11 on: March 16, 2016, 02:21:05 PM »

This is very impressive and the effects in the additives video are super cool.

I remember seeing Quake running on an oscilloscope a few years back (there is a video on youtube), but they weren't experimenting as much with the uniqueness of the medium.

Looking forward to seeing more!
Logged

Quarry
Level 10
*****


View Profile
« Reply #12 on: March 18, 2016, 07:40:29 AM »



Logged
Pishtaco
Level 10
*****


View Profile WWW
« Reply #13 on: March 18, 2016, 07:57:40 AM »

These are all absolutely fantastic WTF. And each one gets better. You have got me looking for oscilloscopes I can buy around here (they all seem to have grey screens - is yours black, or is it exposure settings/low ambient light?)

Can you say a bit about the technical side of what you are doing - is there some analogue of framerate, or triangles per second? Are you just repeatedly drawing the shape you want, and relying on persistence of vision, or is the decay-time of the phosphors or something relevant? What frequency are you synthesizing the sound at, and what happens if you change it, or do some audio filtering on the signal? Does this all give you some limitation on what you can draw, as in e.g. the number of sharp angles on screen at once? What does pure noise look like? Can you change the brightness of the beam?

Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #14 on: March 19, 2016, 12:59:10 AM »

I've been taking a break from stuff for the past week since the most recent problem has posed really difficult for a programming noob like myself. I started cleaning up the code so I wouldn't feel as ashamed to share it with a few people who've shown interest and offered to help. Currently having a ton of difficulty trying to figure out how to constantly populate an array or list exclusively with entries that match certain rules (IE culling offscreen coordinates), so adding new coordinates when they return back into the view, and removing any that leave the view. I tried one loose idea to do it but it ended up being super buggy and didn't work with the animation smoothing and often got stuck if there was too much change on a frame update, so I'm back at the drawing board.

After that, I'll need to see about simplifying geometry, since unity always triangulates all faces which poses a lot of issues by eating up precious refresh rate by polluting the signal with unnecessary extra geometry. That I think I might definitely need some outside help with, but I'll keep trying to slap things together for it on my own since amazingly I've been able to get this far without any outside help, which is pretty big for me.



@Quarry that and a few other things were kinda inspiration to work on this since it's awesome seeing proof of concept of this existing, though I've had the idea since around 2011, but I haven't really been able to do something like this until recently.


@Pishtaco thank you! The most recent video I posted actually covers a lot of those questions, though there's no commentary so I'll explain, and provide times for anything that's covered in the progress clips video.



Analog scope screens are usually grey (or blue or whatever) in normal light, just like a lot of CRT TVs, but the rays compared to that are extremely bright. This is just the phosphor on the inside of the tube, and it's really only very visible in bright light. Turn down the lights, and the contrast between the lines from the display and the phosphor coating is stunning. Some displays will have a darker phosphor, and the vectorscope I use is pretty decently dark, but as far as I know it doesn't seem to affect the image quality much, and most of them are somewhat grey. You could probably get away in daylight by adding a UV filter and turning up the brightness, but it's pretty unnecessary tbh.



The refresh rate is completely flexible. It all depends on the amount of complexity on the screen and how fast you're drawing each frame, and from what I hear, having sound cards that can support higher sample rates can help with getting more precise complex geometry without sacrificing framerate, but I haven't tried it for myself so I can't tell yet. The geometry updates 60 times a second, but I have a smoothing script so that it updates coordinate changes every single sample, since otherwise it looks super jarring. This makes things look super super smooth and absolutely gorgeous in person. Also to clarify, yes it's just drawing each frame over and over again.

In the end, the framerate could be interpreted as {samplerate[48000Hz] / (((complexity - disconnect points) * delay algorithm speed[usually i keep this at around 30 samples per line]) + disconnect points)}. I haven't exactly documented limits yet since I haven't felt that's important so far, but you could use that equation to figure things out pretty well.



It doesn't rely on persistence of vision (like lasers), but rather the decay time of the phosphor. There's an example of this in the video at 0:43 where it slows down from 120fps->30fps on a high speed camera. This shape is already too long to draw completely on it's own, but it does a good job illustrating phosphor decay. Most of the time, the full image is successfully lit and visible on varying shutter speeds, which is shown in slow motion at several parts of the video, where the decay is more visible but the image does not flicker even at such a slowed down speed.

Changing the frequency to something slower (or rather, lengthening the amount of time it jumps evenly between points) adds accuracy and sharpness to the lines, but brings down refresh rate, and also causes the screen to constantly center the majority of the voltage to the center, which distorts the overall shape when it's refreshing so slowly and it keeps moving the recent parts toward the center, making things jiggle awkwardly while being really flashy. On the other hand, having the delay be too quick causes the electrons to only go to exactly the points given and does not draw lines between these points since technically they only exist because of the delay algorithm. On top of that, they create a ton of weird round curves everywhere on the way that become visible only because they're being redrawn equally as much as everything else. When rendering disconnected geometry, 0 delay coordinates are used only because these curved lines happen so fast compared to the rest of the lines being drawn that it decays before it's noticed.

An example of this can be seen over at 3:37, where I change the value of the delay, in which the algorithm spends more samples drawing straight between each point.



Pure noise is actually pretty boring, but when it's combined with an existing signal of geometry, it looks awesome. There's a few examples throughout the video, namely one with the square and the circle jumping everywhere, and it's really cool when the noise (or more often, a controlled audio function) is out of phase with the signal you're effecting and it affects different points at different amounts, making some really mesmerizing lissajous-like stuff.



Lastly, the beam can only be controlled on displays with a Z-axis. Unfortunately, I don't have one, since I'm not exactly very great on funds, and I was lucky enough to get even the single display that I'm working on, but I really hope to upgrade to a display with a Z axis, and the code would be really easy to adjust for such a thing. Doing this would allow for much cleaner disconnects on geometry, as well as some interesting effects with fading lines and stuff.



I'm thinking of starting up a donations page for this project since I've been wanting to upgrade a lot of equipment, but I'm not sure if there's enough interest to warrant begging for pennies.
« Last Edit: March 19, 2016, 01:13:26 AM by ompuco » Logged

Pishtaco
Level 10
*****


View Profile WWW
« Reply #15 on: March 19, 2016, 02:06:27 PM »

Thank you for the detailed reply. I think I get it now - you are running up against the limits of the DAC, not of the oscilloscope. Excepting the inability to turn the beam on and off.

About that -- what if, when passing between disconnected points, instead of doing it in one sample, you put one extra sample inbetween, in a random direction offscreen? In successive redraws these lines will probably not overlap each other, so maybe they will not be visible.



So in the picture on the right, you are just sending the endpoints of the lines as audio samples, and the DAC is constructing as best it can a smooth signal from that, hence the (rather attractive) curves. In the picture on the left, you are interpolating extra points between the endpoints before sending it to the DAC.

You do have some control over the brightness of a line, by changing the number of points you interpolate, i.e. the speed of the beam. It would be interesting to use this to draw a raster image, although I guess at 60fps, and 48000Hz, you have less than 1000 endpoints to play with, so say a 30x30 resolution.   
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #16 on: March 21, 2016, 01:56:57 PM »

@Pishtaco I'll have to give that method of disconnected geometry another try, though it might prove problematic by changing the center balance, and it would also cause some issues when the next piece of geometry starts since as you notice there's some shaky lines where the speed changes, and that'll be amplified when it's coming from further away. I'll see what happens. I tried something similar, having it change offscreen position for every sample there was a piece of disconnected geometry, but it got kinda messy the way I implemented it and I got some weird sandy looking artifact.


Also, about rendering raster video, there is an electronics project which does this. It's actually kinda neat.






There's also something Hansi Raber made that's similar that's done entirely on the computer and tries to match the frequencies to an Aphex Twin track Come To Daddy, and works kinda like traditional scan lines which is pretty fitting for a CRT scope.





idk if i'd really care to try it any time soon, but it's certainly doable.

Hansi Raber, Adoxoco and Jerobeam Fenderson have been a rather big influence on me working on this project. Initially I really wanted to try the software they were working on but I grew restless waiting and then just thought I should try making it myself, and it's really proven an awesome way to get into programming and electrical design. Here's a preview from their project that I was not fortunate enough to help kickstart :<

https://www.youtube.com/watch?v=qnL40CbuodU





Anyways, I'm thinking of doing a livestream of new scope work tonight when I get back from classes, so if anyone is interested, I'll post a link when It's happening. I'll be answering any questions on mic as well as be trying some new stuff. Also keep in mind any and all questions are welcome, even things that seem pretty rudimentary (like "WTF is an oscilloscope" or stuff like that, I'm happy to be all chatty about this stuff).
Logged

diegzumillo
Level 10
*****


This avatar is so old I still have a some hair


View Profile WWW
« Reply #17 on: March 21, 2016, 02:32:28 PM »

There was some news a while ago about a codec that renders pixel to vector graphics. Supposedly revolutionary, but I never heard news from it again. Of course, vector graphics doesn't mean "you can run it on an oscilloscope", but it could useful.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #18 on: March 21, 2016, 02:39:23 PM »

There was some news a while ago about a codec that renders pixel to vector graphics. Supposedly revolutionary, but I never heard news from it again. Of course, vector graphics doesn't mean "you can run it on an oscilloscope", but it could useful.

I actually thought about using something like that early on since it would make some stuff easier, but I thought it would be much cooler to work with straight genuine vector geometry. Plus this way I have more control over stuff. There were a few demos for a program called OpenLase which I studied for a bit that did that with video.
Logged

ompuco
Level 3
***


oh my god whats happening


View Profile WWW
« Reply #19 on: March 30, 2016, 12:02:37 AM »

Sorry I've been silent for a bit, been trying to fix up some stuff and improve things. I managed to get a primitive renderer in Unity finally, so now projects are playable without a scope. There's a lot of room for improvement with that, though, so I'll try getting stuff fixed and made for that. Here's a demo of it in action with the room demo and a chorus audio filter, one example of the effects achievable with this render method.

Logged

Pages: [1] 2
Print
Jump to:  

Theme orange-lt created by panic