Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1401637 Posts in 67933 Topics- by 61476 Members - Latest Member: meeloboer

July 03, 2022, 03:59:45 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Using surface ids to improve edge detection in my Unity game
Pages: [1]
Print
Author Topic: Using surface ids to improve edge detection in my Unity game  (Read 712 times)
ianmaclarty
Level 0
*



View Profile
« on: March 10, 2022, 04:49:26 PM »

I recently had an idea for an improvement to the way I render lines in my game Mars First Logistics. I often get questions about how edge detection in my game works and I think it’s different enough from other approaches I’ve read about (notably Obra Dinn and Manifold Garden) that it was worth writing up.

Edge detection is a post process effect where first everything is rendered to a texture and then we read that texture to work out where the lines should go. Here's what it looks like before and after applying the post process shader:



With edge detection we're looking for pixels whose "value" is different from its neighbouring pixels (I'll get into what "value" means shortly). We can then darken these pixels to make lines along the edges.



I look at several values to determine if a pixel is along an edge: its depth (distance from camera), surface normal and colour. This info is encoded in 2 textures: the depth texture generated by Unity and the camera’s 4 channel colour buffer (16 bits per channel).

Colours are stored as an index into a palette in the blue channel. This means I only need to use one channel for colours and it gives me complete control of colours at different times of day. Here’s what the day time palette looks like:



Instead of trying to store the x, y and z components of the normal separately, I store the dot product of the normal with the view direction and another orthogonal direction. These two dot products are stored in the green and alpha channels.

This gives pretty good results, but it’s not perfect. You can see little gaps in the lines where the dot products are not sufficiently different to be regarded as an edge:



Another issue is on curved surfaces, at particular distances, the normals can change too rapidly and the whole surface gets detected as an edge:



The idea I had to fix both these issues was to pre-compute the “surface IDs” of each mesh and use these values instead of the normals for edge detection. A surface here means a set of vertices that share triangles.

This works because vertices along sharp edges of a mesh do not share triangles, so the pixels around sharp edges will have different surface ids. Here’s what the game looks like with all the surfaces coloured differently:



And here’s with edge detection applied. Perfect!



I was feeling pretty pleased with myself, but then I started noticing some weird artifacts on some surfaces, like this:



Using RenderDoc, I tracked this down to the surface ids losing precision somewhere. This is a closeup showing pixels that should all have the same surface id:



It wasn’t an issue with the texture channel precision, because the surface ids are small enough to be exactly represented by 16 bit floats (they’re all integers in the 0-600 range).

The weird thing was even if I set all vertices to have the same surface id in the vertex shader, the values in the fragment shader would be inconsistently different.

I eventually determined that it was the interpolation done during rasterization that was messing up the values. I guess this calculation must be done at a fairly low precision. If I turned off interpolation on surface ids the problem went away.

The problem now was that Unity’s surface shaders don’t support nointerpolation. It was possible to reproduce the surface shader features I needed in an unlit shader (basically just shadows and directional lighting), but this felt like it would be harder to maintain.

In the end a simple round(surfaceid) in the fragment shader seemed to fix the problem. Phew!

I did have to clean up a few of my models where I hadn’t marked surfaces as smooth that should have been, but that was worth doing anyway, if only to reduce the vertex count.



Even with surface ids, I do still keep the dot product of normal and view direction in a channel, because that’s still useful when using the depth to detect edges.

Consider the case where a surface is almost parallel to the camera’s view direction. The depth can change very rapidly from pixel to pixel, leading to false edges like this:



This can be fixed by biasing the depth edge threshold by the dot product of the normal and view direction. If the dot product is close to zero, then we don’t detect edges.



Finally the red channel is used for gradients, which is useful for things like sunsets. I lerp between the colours associated with the colour id stored in the blue channel and the next colour in the palette (color id + 1), using the lerp factor in the red channel. This means I need to put colours used in gradients next to each other in the palette.



Shadows are stored in the sign bit of the blue channel. Here’s my final layout of data in the colour buffer:



A bonus of this approach is the wireframe effect behind dust can be achieved using a simple colour mask. The dust shader only writes to the Red and Blue channels, preserving the surface-ids and normal-dot-view-dirs of the objects behind the dust, while replacing the colour id.



Thanks for reading! I’d love to hear any questions or suggestions for improvements.
« Last Edit: March 10, 2022, 05:11:38 PM by ianmaclarty » Logged
InfiniteStateMachine
Level 10
*****



View Profile WWW
« Reply #1 on: March 10, 2022, 06:41:51 PM »

Wow nice work! That was a great read!

Definitely one of the best outline effects I've ever seen. So much better than what I'm using on my project haha

I do have a question regarding the surfaceID stuff.

Basically my question is if it's interpolating the surface ID value across the surface of a triangle and the values across the triangle are all the same, why does it get noise?

Logged

ianmaclarty
Level 0
*



View Profile
« Reply #2 on: March 10, 2022, 07:01:57 PM »

Thanks!

That's a good question and was why I was so confused by the noise too. I decided it must be the interpolation, because when I made a test unlit shader, and added the nointerpolation modifier to the surface id, the problem went away. I think the issue is that because the interpolation takes into account perspective, it's not just a simple lerp and probably does some division too (I'm not sure what the exact formula is). I'm also guessing that the GPU doesn't have a special code path for when the vertex values are the same, so depending on the precision it uses, applying the interpolation formula could result in a slightly different value, even if the vertices have the same surface id. It'd be great to get the opinion of someone who knew more about this kind of low level stuff though!
Logged
InfiniteStateMachine
Level 10
*****



View Profile WWW
« Reply #3 on: March 10, 2022, 07:19:44 PM »

Interesting! I need to see if I can dig up some white papers on how interpolaters work under the hood. I kind of always just took them for granted haha

With regards to the surfaceID stuff. Is that data you setup or is that a builtin value? I seem to recall that maybe unity has primitive id's in their newer pipelines but I could be 100% wrong about that.
Logged

ianmaclarty
Level 0
*



View Profile
« Reply #4 on: March 10, 2022, 07:25:56 PM »

With regards to the surfaceID stuff. Is that data you setup or is that a builtin value?

I generate the surface ids on model import. It's basically just dividing up the triangles into connected sets and then assigning different integers to each set.
Logged
InfiniteStateMachine
Level 10
*****



View Profile WWW
« Reply #5 on: March 10, 2022, 08:21:41 PM »

ah gotcha, super interesting method Smiley
Logged

Prinsessa
Level 10
*****



View Profile
« Reply #6 on: March 11, 2022, 02:56:14 AM »

Gorgeous, thanks for this writeup! Kiss
Logged

Thaumaturge
Level 10
*****



View Profile WWW
« Reply #7 on: March 11, 2022, 07:53:24 AM »

This was a rather interesting technique, and write-up thereof--thank you for sharing it! ^_^
Logged


A Fantasy Top-Down "Space" Metroidvania
Itch page ~ Twitter
buto
Level 0
***



View Profile WWW
« Reply #8 on: March 11, 2022, 01:11:58 PM »

The screenshots look absolutely gorgeous! Thanks for sharing your ideas and implementation details!  Coffee
Logged

Pages: [1]
Print
Jump to:  

Theme orange-lt created by panic