next »

[Zorg]

Let's assume i would only use unlit materials without any texture in my 3D scene.

What is the easiest way to check if the output of the main camera is only one single color/material?



I don't need to be able differentiate between the following cases, but i neeed a function which covers all those cases:
1) The camera is pointed directly at a wall which uses only one material.
2) The camera is pointed directly at a wall which uses multiple materials, but all polygons facing the camera share the same material.
3) The screen is filled with multiple objects sharing the same material.
4) The screen is filled with multiple objects using multiple materials, but all polygons facing the camera share the same material.

[nova++]

What specifically is the context for this? Is the only way to achieve what you wish to directly look for materials?

[Zorg]

Thank you for asking.
I "just" need to know if the whole screen is currently single-colored (checking at all times).

[...] Is the only way to achieve what you wish to directly look for materials?

I have no idea how to approach this problem. I thought looking for materials might be an option. For example: if i would fire two Raycasts in a random forward direction (in camera view) and the materials of the RaycastHit objects are different, everything is fine because i know the screen is currently not entirely single-colored. But it gets complicated if i have to keep firing more and more RayCasts because all of them returned the same material.

I want to know if i'm missing an obvious (hopefully simple) solution?

[oahda]

If the only condition is that they should be the same colour (and you don't need to differentiate between different materials that share the same colour), you could look at all of the pixels of your framebuffer (probably a downscaled version or it might get expensive; should work fine anyway unless you need to catch absolutely minute stuff) to see if they're all the same.

[nova++]

If the only condition is that they should be the same colour (and you don't need to differentiate between different materials that share the same colour), you could look at all of the pixels of your framebuffer (probably a downscaled version or it might get expensive; should work fine anyway unless you need to catch absolutely minute stuff) to see if they're all the same.

Seems like something that could be done with GPU compute, perhaps? Maybe I'm overthinking it :p I've been on a GPU compute kick lately so you know what they say about problem-solving when all you have is a hammer...

[oahda]

I was thinking of the CPU but I've also yet to touch compute shaders. Whichever way gets the job done!

(but compute shaders would reduce your potential audience since they don't work everywhere…)

[Zorg]

If the only condition is that they should be the same colour (and you don't need to differentiate between different materials that share the same colour), you could look at all of the pixels of your framebuffer (probably a downscaled version or it might get expensive; should work fine anyway unless you need to catch absolutely minute stuff) to see if they're all the same.

How to access the framebuffer? I read some suggestions about creating a RenderTexture (2nd camera, low resolution) and use ReadPixels to Texture2D and use GetPixels. But others wrote it would be too expensive (especially to use GetPixels continuously).

[nova++]

How to access the framebuffer? I read some suggestions about creating a RenderTexture (2nd camera, low resolution) and use ReadPixels to Texture2D and use GetPixels. But others wrote it would be too expensive (especially to use GetPixels continuously).

A separate camera and render texture would work, but it also would double the amount of drawcalls happening and have a performance hit. Maybe not a big hit, but a hit nonetheless. On the other side at least, with a sufficiently small texture (like, 32x32?) you might be able to just foreach through the list of pixels without much performance hit.

Really, it's something that's going to need some iteration and experimentation.

[Zorg]

Found this article https://medium.com/google-developers/real-time-image-capture-in-unity-458de1364a4c which points at https://docs.unity3d.com/2018.1/Documentation/ScriptReference/Experimental.Rendering.AsyncGPUReadback.Request.html in an update note. I can't say i'm understanding the article completely, but it looks promising, i guess.

[whendricso]

The simplest way is to grab the screen from a render texture. Like some others said it's probably better to check a lower-res one unless you absolutely must check every single pixel.

First you need to output the screen to a render texture. It might be faster to actually have a separate camera render a lower-res texture than camera.main but you'll need to experiment to verify this. Next, you can just loop through all of the texture pixels to see if they match. The simplest way is to grab the color from the first pixel and store it's rgb values. Then, you only need to check the G and B values if R matches the previous pixel.

[Zorg]

What you described is the "the naive approach", described in the linked article in my last post. The comment of the author is:

And done! Do this on every frame and performance is also done! In fact, if you’re building a VR experience, you can’t do this even once.

Here are the underlying reasons this is slow:

• GetPixels() blocks for ReadPixels() to complete
• ReadPixels() blocks while flushing the GPU
• GetPixels() allocates a new array on every call, thrashing the garbage collector

It's the first thing i'll try with a 2nd low res camera nonetheless. Did you try this before?

[ken]

How are you using this knowledge elsewhere? If it's to feed another shader, maybe you could just use the stencil buffer? Curious about the end goal of this, if you're able to share. (Just wondering if there's an 'XY problem' here.)

[Crimsontide]

How are you using this knowledge elsewhere? If it's to feed another shader, maybe you could just use the stencil buffer? Curious about the end goal of this, if you're able to share. (Just wondering if there's an 'XY problem' here.)

Yeah... this feels like an XY problem.  I can't think of a situation in modern games where testing the entire screen to ensure its a single color would be the most efficient way to solve any problem.  In fact the only thing that comes to mind was the old NES Zapper and duck hunt.

[whendricso]

Not efficient, but easy to write

[oahda]

Slow for sure. But easy. So if the game is simple enough that it can handle it, that would be nice.

I would be wary of suggesting some general solution tho since it probably depends heavily on what the actual gameplay is like and if there's any restriction on possible camera angles and so on… Would be great to know more about the game, perhaps a screenshot too?

For example if you were to test it by looping over triangles and checking their facing direction in relation to the camera, maybe you would have to get clever if you have a lot of them, but again, striving for simplicity where possible, if your scenes are tiny maybe you could just brute force your way through all of them at a decent speed.

[Crimsontide]

I still want to know WHY!??  Simply for the sake of curiosity.

[Zorg]

Haha, thanks for the interest in this topic. (I won't have time until next weekend to even test the simplest solution.)

Eventually, i'll use it as a little effect, i want to manipulate the player's position at exactly the moment the screen is painted in a single color, that's all.

On top, i'm interested in the question. I did not think it would be that difficult to find one-colored frames, so i had to ask because i did not find a simple solution.

[Daid]

Sounds a lot like one of the effects that Antichamber throws at you. Where looking trough an window fully teleport you to a different room.

Pretty sure they don't look at the screen, but just have it setup for a certain viewport to match the window. If your effect is quite limited in use location, I would go with that route instead of checking the screen.

If there are a lot more scenarios where you can secretly move the player, then you should consider how well that will actually play... as it will be confusing for the player.