next »

[gimymblert]

Those dolphin HD image actually show BRUSH STROKE
I wonder how they did this shader, the wii is not too strong to analyze an image, and that's an insane amount of data to match 3D point  
They didn't go half way about painterly look, it's reallly painted

I'm willing to pay anyone who can replicate this 3D shader  

Must work like it would on wii real time, must be possible on unity3D free at best
Price to debate on result

[Trevor Dunbar]

I was able to get a similar effect using some filters in PS: smudge stick + angled strokes or Rough Pastels.

Thinking of how you'd do this in a pixel shader right now.

The brush stroke looking texturey stuff going on would be the actual canvas underneath.

It looks like any geometry that isn't the landscape is just shaded normally...that's really cheap. It makes it look like those things don't fit in.

It looks almost like the pixels were blurred than blended with the input pixels

[Nix]

someone on these forums wrote a tutorial: http://rada-game.blogspot.com/

[gimymblert]

http://www.youtube.com/watch?feature=player_detailpage&v=clWznfxUflo#t=564s

I have been looking to this HD video to understand the effect, it's pretty early in the game (right after zelda fall I think).

Of course the effect isn't a simple shader, it's multiple effect combine together.

Obvious one seems to be premade object ribbon and "static" particle (tied to object mesh)
There is also a bit of edge detection (backfacing triangle edge share with  visible triangle), A bit of LOD (fade and replace by particle after a distance) A bit of a scanline like effect for texture (not sure).
And some magic obviously through post processing of screen AND texture.

But it does not explain everything.

I don't get all of it, it's pretty complex. Some part I don't explain it at all, also very great temporal consistency.

Also obvious art rules applications (light seems to be above darker colors, rules bases on object locality, etc...)

It's one of best NPR rendering I had ever seen.

That said the character are rendered differently not because of cheapness but to make them pop, they use to have the same shader but they blended too much with the background.

@Nix
It's not quite the same effect (well there is this one but there is another), if you look closely the brush stroke are not "filled" stroke, but you have a kind shape on the brush like a simpler:

That's why I post here, only for this brush effect

edit
Also isn't the wii too weak to do such large scale bokeh effect?

[Trevor Dunbar]

http://www.naixela.com/alex/

http://www.naixela.com/alex/downloads/gs_painterly.zip

[gimymblert]

requires: shader model 4 graphics card

Not as cheap as running on wii and does not work on my computer 

[Trevor Dunbar]

Ok, what shader model DOES your card use? And what format would you want the shader in if I can ever get it figured out?

[_Tommo_]

Uhm, I am wondering too how they could achieve that on wii... naively it looks like some uber-complex fullscreen deferred 3.0 shader.
Instead it must be some kind of hack.
I think we miss something about wii's internals: the wii is known to have built-in hardware effects that are somewhat heavy on other architectures, like DOF or fullscreen blur.

From the images, for sure it is not a fullscreen effect like those in the links: it's obvious because
-the smudge brush never alters polygonal outlines
-the smudge is different depending on the texture
-the smudge is content-aware (ie: is directional on leaves) and much, much more higher quality than in those experiment
-the smudge is depth aware: it smudges distant things more
-moving objects aren't smudged

Doing this at fullscreen would require some deferred depth and normal rendering + a crazy shader, and throwing tech at the problem has never been Nintendo's style (and wii can't possibly support something like this), so I think that our shader 4.0 guys are out of track.

My bet would be on a dead-simple linear interpolation between a sharp texture and a pre-smudged texture based on pixel depth, something as simple as

Code:

float d = max( 1,  pixel.z * DEPTH_MULT );
out.color = tex2D( base, uv ) * d + tex2D( smudge, uv ) * (1-d);

And that looks like something the Wii can definitely do in a Fixed Function unit.
Given lots of artist effort (two textures for everything) it allows lot of artistic control, nintendo style.

[Dataflashsabot]

Given lots of artist effort (two textures for everything)

I'm sure it would be easy to have a tool that creates and updates the smudged versions automatically.

[_Tommo_]

I'm sure it would be easy to have a tool that creates and updates the smudged versions automatically.

But if you look at the images, this is clearly not the case, as each texture has a different smudge... leaves are quick radial brushes, grass is blotches of color, architectural features are a single strong brush (look at the thing over the door temple).
So I think that the "brush texture" is probabily handmade.

Tool = bland, and Nintendo does not like that 
But yep, you could sure run all the textures through a Photoshop batch action and have it smudge everything.

[Ishi]

I did something similar once. I don't have the output any more or any quick way to implement it at the moment, but I can explain it.

I started with a texture with R going 0-1 horizontally, and G going 0-1 vertically. Then I applied a brush stroke filter in Paint Shop Pro to get this:



Let's call this the Filter Texture.

Then:
- Render the scene to a texture (Scene Texture)
- Render the Scene Texture to the screen, full screen, but:
  - For each pixel, instead of sampling the Scene Texture at (u, v)..
  - Sample the Filter Texture at (u, v). This gives you an R and G value
  - Sample the Scene Texture at (R, G) and output that pixel.

It's not perfect because it does remove a lot of detail from the image, because some of the pixels in Scene Texture won't have any effect on the final image. Also the brush strokes don't follow the contours of objects. Maybe the Filter Texture could be generated on-the-fly too based on the depth buffer.

--

Another thought; could Skyward Sword be doing something simple like manually-generated mipmaps? The lower size mipmaps for the trees and stuff would just have very broad brush strokes, and as you get closer to them they'd resolve into the full-detail textures. I would expect Nintendo to come up with some sort of simpler-than-you-think solution like that.

Edit - actually that's what Tommo suggested above. Didn't read the thread properly

My bet would be on a dead-simple linear interpolation between a sharp texture and a pre-smudged texture based on pixel depth, something as simple as

Code:

float d = max( 1,  pixel.z * DEPTH_MULT );
out.color = tex2D( base, uv ) * d + tex2D( smudge, uv ) * (1-d);

And that looks like something the Wii can definitely do in a Fixed Function unit.
Given lots of artist effort (two textures for everything) it allows lot of artistic control, nintendo style.

You can achieve that with mipmaps basically for free because they're so ingrained in the hardware.

[_Tommo_]

That's right, you manually provide the mipmaps and you get the effect for free!
Nice find, I guess we have it now  

Nintendo guys always are the finest source of graphical hacks, they really know how to get art out of outdated hardware 

[gimymblert]

To fuel the discussion a bit more (maybe this should be move to technical now )
This early promo shoot show early implementation, look at the artifact on the ground near the stone





About the mipmap it's unlikely, the brush have great temporal consistency, and behave like non depth and angle sensitive billboard (a 7 dotted brush) that seems to pick the underlying pixel, the effect reappear on tree when they ramp it up (bring it closer to the camera) on combat. Brush stroke is likely to be an artefact of multiple brush of the same color side to side and overlapping except for those manually place on mesh to emphasize form, the whole deal is to separate those who are manually place (look at the video and you see instance where it's force compare to other brush), generate on mesh data (look at the video I provide when link cut a tree), and procedurally generated.

[rogerlevy]

It looks to me like simple particle-based sprites.  You can tell looking at the grass which has individual brush stroke sprites that appear to be floating just about the ground and stick out past the edge of the cliff polygons.  The shader might generate them automatically (so that designers wouldn't have to manually place them, and to save storage space).  Using a special vertex shader this could be extremely efficient, maybe storing just x,y,z,normal,brush type token (for texture UV selection),rotation angle,and color (for variation). 

The trick I think is having an independent scale factor so that they don't scale normally with everything else, and fading out or clipping (using stencil etc) the ones that are not attached to polygons facing the camera.  The same texture could store all the brush strokes and an integer on each one could be used to tell the shader which one to use and the effect could be done in a single function call.

[gimymblert]

The wii don't have a shader (or better it's fixed point)

[Farbs]

Darnit. I was working on something like this just the other week, and was hoping to be first 

Here's how I think they're doing it:

• Render the scene to a buffer using big chunky polys as per usual.
• Textures should contain contrasting colours and a bit of noise so as to create visible brush strokes.
• Use stencils (if the Wii has 'em) to mask out unpainterly areas during primary render.
• Finally, render the prerendered scene to the display buffer seven times offset using something like the following pattern, and using a blend algo that only picks the brightest of the source and destination colours. You can see halos around the upper leaves where the brighter green for the leaves meets the brighter blue from the sky, so both the blue sky and the green leaves appear on the display. A cleaner (but costlier) implementation would calculate a single brightness value from all of RGB and use that for the comparison.

After this is applied elongated areas of consistent bright colour explode out into brush strokes, drawn over any darker background.

So...

...becomes...

They may be doing other fancy things to clean this effect up, but I'm pretty sure this is the basis of it.

[Farbs]

Here's a prettier example

->

[Farbs]

...and here's my quick implementation in UE3.


The leaves are a little too contrasty to work, but the rocks in the top left come out nicely, and there are some clear brush strokes along roots/branches/trunks.

Sampling off pixel boundaries would smooth it out a little, and I think I have the pattern a bit wrong, but the general effect is there. I can't see any way to do this in non-pro Unity unfortunately.

[_Tommo_]

That's utterly awesome 

[bart_the_13th]

...and here's my quick implementation in UE3.


The leaves are a little too contrasty to work, but the rocks in the top left come out nicely, and there are some clear brush strokes along roots/branches/trunks.

Sampling off pixel boundaries would smooth it out a little, and I think I have the pattern a bit wrong, but the general effect is there. I can't see any way to do this in non-pro Unity unfortunately.

That's ow... My jaw dropped...
The leaf probably because you use model with small leafs, I guess it will work better with big leafed trees, or most likely mushroom styled trees.

BTW, the water-color-painting shader supposedly works as DoF shader, so it blurs(in painting way) object at distant.

I havent tried it, but maybe, just maybe, it can be simulated using projected texture in Unity3D Free... Just project a crosshatching/water painted texture to the specified models from camera position, maybe adjusting the blending mode in process?