Just fighting boredom, I wasn't expected to actually implement it, it's just a thought experiment in lateral thinking. But now I'm curious about actual practicability
You do stuff much more complicated anyway, I read them all the time here lol
Raytracing (at least like I present it here, which is more accurately raymarching) is dead simple though, you just look into a grid a value, if it's the right value, proceed into the next relevant cell (in the direction of the ray), else get the position of the grid or get the value of the grid. On teh gb Thing I find a way to do it faster at bit level using bit tricks.
Else it's basic math of line primitive-intersection, you loop the primitive against a ray and take the closest hits, do it for each pixel of the screen. Anyone can get a raytracer under 100 line of code.
The real thing with raytracing is less about the raytracing part and more about containing the explosion of resources needed for GI. Basically for each hits you need to generate reflective rays, and they are a hemisphere of point above the hit. So for every hit say you need 1000 more rays, these ray will hit thing and generate 1000 other rays ... oups ... The code is teh same, you just add more rays to loop in.
The other difficult thing about Raytracing is still not raytracing, it's the "light transport" ie how much energy ray have in a direction after a hit, which is based on the brdf (shader) of the material hit.
The other difficulty is still not rayTracing, it's about how do you spend your ray budget to get a cool image. Even with many rays you have sampling issue and the image get noisy, the more rays the better but slower, so you need strategy to get more from less, such as importance sampling (concentrate rays where it matter the most, usually based on the brdf), denoising (remove the noise appearance by guessing how smooth the result is based on the noise and a few heuristics).
The other difficulty, still not raytracing, is how to loop efficiently all objects without having to loop through all of them and discard those we will never hit. Which is like spatial partitioning, you can use voxel type or tree type of structure to query object in ray path. It's the same in physics engine.
On gameboy I was thinking only about the first hit to detect very wide area on teh screen (not pixel) with shallow number of elements, all based on a small 2d grid. Based on the size of bytes.
On the other projects with cubemap, I simply consider a screen as first hit (that's what an image is, all the rays that are reflected back to your eyes). SO I simply take a picture of all objects with their UV colors, from the cubemap position. Since for every pixel, in shader we use the normal to find the color on the surface of the sphere of the cubemap, instead of using that color directly, I use it to get the color at that UV position defined by the color, like in flow map or glass shader, if you ever made those kind of distortion shader you have half the work done. Then I mix that with other techniques (box projection, dynamic texture) to simulate sampling above each points (that is I take multiple samples by shifting the normal 90° around it's direction in a circle) which I then accumulate each iteration in the dynamic texture. There is some more shenanigan (like properly weight the accumulation by distance, angle from the original normal, etc ..) but they aren't hard.
On the cubemap thing, I also use the lightmap as the lighting surface directly. Generally you write a shader, and in the shader you write how the light interact with the point. When you use lightmap, you skip the lighting and just sample the lightmap. But what if you do lighting directly in teh lightmap? That's a lightmap Gbuffer, basically I store the normal (in world space), the shadow masking (there is many way to do that) that will block the light. Then for each pixel I find the light that I store in a dynamic writable texture (light accumulation) which is teh texture running the shader and taking the Gbuffer lightmap as argument.
For the shadowmasking, I have many options, I store the XYZ positions of the pixel (precision issue with 8 bit), so I can compare with the shadowmap. The result is used to mask the usual light computation with the normal and the light position. Another option (the hard one) is to bake shadow with a Spherical harmonic texture, if we need time of day, instead of using a sphere data, we will use circle as time of day is generally circular, which mean less data to store. The simple way is just to have a regular another shadowmap texture and read that to attenuate lighting.
That is the whole operation is from the dynamic texture:
1 - read the Lightmap Gbuffer input (normal [RGB], world position [RGB], shadowmask [A], cubemap index [A])
2 - compute lighting with the normal and shadowmasking (like in regular shader) of the Gbuffer
3 - sample the UV cubemap atlas texture using the index stored in a channel of the Gbuffer
4 - use the cubemap sample to read the Gbuffer again
5 - compute the GI light of that sample
6 - mix and store all the data into itself (accumulation)
7 - Repeat next frame/iteration (can be frame independent)
A - Shader in static environement, read the lightmap, done!
B - Shader in dynamic objects, read the box projected UV cubemap with normal, sample the lightmap using teh UV data, DONE!
Another way is to separate the lighting calculation (1 and 2) in another dynamic texture (direct lighting texture, basically automatic baking), so we sample the light directly instead of calculating each frame (light probably won't move) and each time we sample a GI light (5).
Open GL es 2.0 are limited to 8 texture samples:
1. 2 textures samples RGBA of the LM Gbuffer
3. 1 textures sample of the cubemap per GI sampling
4. 2 textures samples RGBA of the LM Gbuffer per GI sampling
- That is 5 textures samples out of 8 for the simple case, and 3 textures samples per "rays", that is 2 rays max in OGL 2.0 But I have omitted the albedo sampling in the GI pass, so that's 4 sampling per ray with it. ie only one ray if we want colored GI with 2 extra remaining unused sampling.
- With the separate Direct light texture, we still get 2 textures samples per GI pass, since we need the position (for attenuation) on top of the lighting, but since we separate the light pass, we can get the color back since it can be done in the direct light pass (3 samples per direct update: normal+shadowmask, position and albedo) but we loose complex light transport (which is fine on low end).
Some optimizations (quality or speed) note due to the flexibility of the shader, which mean we can scale it as complex as we want/need it to be. I'll make bold the simple version, anything else is complexity build on top and probably not needed for the a simple working version, so if it's not bold you can probably skip it.
in 4 - it's using a
a simple approximation technique call box projection (google it), and basically the volume that define the space of the projection is how we get the index per point,
that is when the volume overlap or is close to a point in the lightmaped triangle. We can probably store more than one index and interpolate in order to have less artifact. Deciding how many cubemap sample per point and how to store the index seems like a way to experiment to improve the final render. How we store the cubemap atlas is also another point of flexibility, the most simple and intuitive is to
use a lat long 2d cubemap (indexed by the normal x y and z) but isocahedron seems to be more efficient, it seems simple but still need to google the formula.
in 5 -
That's the most involved part, it's not necessarily complex, but
there is a lot of flexibility to scale complexity there. Basically that's where the light transport GI happen.
* The first thing is that we can sample a single point per frame, advancing the loop at the next frame to sample the next (parameter passed by a script to the shader). Useful for low end open gl 2.0 hardware who has a limit on how many textures sample we can have.
Or get many points in a single loop pass of the shader on high end machine.* We can do some importance sampling optimization probably somewhere in the sampling loop to reduce number of samples needed and ameliorate the quality, no idea how to do it, but that's something we can do ...
*
Ideally we should recalculate the whole shader of the sample points (ie sampling the normal, the shadow, the colors (albedo) etc ..) to get proper lighting, calculate the attenuation using the two position to get the distance, calculate the angle attenuation and specular contribution by taking the vector toward the sample and doting it with the normal of the sample, and then weighting the result by taking the normal of the point with the direction toward the sample ...
BUT THAT'S TOO MUCH WORK,
we would just sample the direct lighting (by using the normal calculation OR sampling it with the "direct light texture" idea) and attenuate it with the distance to the sample.*
If we have a direct light texture, we can use the mipmap to sample the average colors of close points, that would count as a multiple samples,
but beware of the edges since we are on a lightmap representation. It's similar to cone tracing in voxel GI, thus we can basically also select which mip map to sample, by using the distance and computing the coverage to get the cone size giving the correct equivalent mip depth. However this increase light bleeding from sampling incorrect data from neighbor triangles that might not have the same direction. There is probably some research to do to limit the bleeding, probably with comparing the normal of the samples or extra data in the texture?
* Since we have the position of the start and end point (point position and sample position) we can use that to intersect a list of primitive in high end machine, and return the data if interception instead of the sample data.
Edit: due to the technique being lightmap based, it's limited to object being in the lightmap.
I have the idea of reserving cubemap index 0 as the "farfield" or basically the "skybox of the region". This cubemap is accessed for UV points in the sampled cubemap that have data (0,0), it's basically every other region outside the current one. If we calculate other region then render other region from that farfield cubemap, we can get their lighting contribution sampled instead of getting the sample from the Gbuffer. By updating region in specific order we can get further, then all the farfield cubemap we get distance lighting contribution too. Those farfield region can also be LOD.
Instead of storing light accumulation into pixel colors, we can store them into SH to have directional, but going from 3 channels (single texture) to 3*3=9 for SH1 (3 textures) or 3*9=27 for SH2 (9 textures))
Using bent normal would also ameliorate the result, bent cone are cone that are in bias toward the occlusion average, ie where most light would enter if the occlusion above the point is convex. this something an external soft will bake anyway.