Game Dev Blog & Tutorials

Sprite Stencil Overlay Breakdown


Hey! I recently helped someone out with an effect they wanted to create, which involved creating an dithering-like overlay effect when the character is behind other sprites, (similar to the example shown in the tweet above). I thought this might provide a nice example of how to use the Universal RP’s Forward Renderer to override Stencil values.



Layer Setup

To start with this effect, our Player GameObject (with the SpriteRenderer) needs to be assigned to a new Layer, which we will call “Player”.

Objects that are in-front of the player (where the overlay will appear) will also be assigned to another layer, named “Infront” (or “Foreground”, the name isn’t too important).


Example Overlay Shader

Next we need to write a shader for the overlay effect. I’ve created a new Shader Graph using the 2D Sprite Unlit Master node. I’d like to create a dithering-like effect which could be achieved using the Dither node. I don’t want it too small though and it uses screenspace pixels, and the “pixels” in my sprites are actually made up of multiple screen pixels. Therefore, I’ll be using a checker-board approach instead. There is a Checkerboard node, however this is anti-aliased, so will have transparent parts, which may not be the intended result if using a pixel perfect approach. Instead, I’m creating the checker pattern by using a combination of Fraction, Step and Lerp nodes – (but play around with each approach and see which suits your project best!)


Can use Position set to World rather than UVs if you don’t want the effect being stretched & scaled by UVs. (Note, the twitter gif also uses Add 0.5 after the Lerp).

To use the sprite from the SpriteRenderer component, we use the texture property with the _MainTex reference. Since this is just for an overlay, I only need the alpha component of the texture, which is multiplied with the checkboard effect to ensure it stays within the bounds of the sprite.

We need to create a new Material using this graph. We’ll be overlaying this material on top of the player sprite, but only when it is hidden behind objects in the Infront/Foreground layer. This can be achieved by using the Stencil overrides on the Forward Renderer’s RenderObjects feature.

Forward Renderer


Before going into the stencil overrides, we need to make sure the Forward Renderer is set up correctly. You should have a Universal Render Pipeline Asset object, somewhere in your Assets. If you used the URP button when creating the project, there may be multiple pipeline assets, named like “UniversalRP-HighQuality” under the Settings folder, you’ll want to make sure each quality setting is using the same Forward Renderer. If you don’t have these, you’ll want to go through the URP set up as shown in the docs.

The Forward Renderer has a box to add Renderer Features to the pipeline. Unity provides a feature known as RenderObjects, which re-renders any objects specified by the Filters on the feature, but also allowing certain things to be overriden, such as Depth, Stencil and Camera projection. We’ll only be focusing on the Stencil overrides.

As we don’t want to render the Infront layer objects twice however, we should also exclude them from the Default Layer Mask (split into Opaque and Transparent Layer Masks in newer versions), at the top of the Forward Renderer. This means we won’t render objects normally on that layer, so in the scene/game view those objects should disappear. But we can then add a RenderObjects feature to render them, specifying the additional overrides (We’ll be going over this in a bit).

For the Player layer, we want to render it twice so should keep it in the default mask. This is because we will be overriding the material to create the overlay and still want the normal sprite/material to show.


Stencil Overrides

The Stencil overrides allow us to write a value to the Stencil buffer (for shaders this is usually an 8 bit integer value which is 0 to 255, however the Stencil Value slider only allows values of 0 to 15… for some reason. We only need to use a single value for the effect anyway though).

We can also test against the value currently in the buffer at the same time, based on operations like Less, Greater, Equal, Not Equal, etc. If the test fails, it discards the pixel, and the Pass, Fail and Z Fail settings show what happens to the value in the buffer when the test passes or fails (e.g. Keep the current value in the buffer, add/remove 1 from the current value, or Replace it entirely with the value that was tested).

For more information on the stencil buffer and operations, see the Stencil docs page, (note that it is for code-written shaders but still applies).


Render Infront Objects

So in order to create our overlay effect, we first need to have our Infront layer objects write a value of 1 into the Stencil buffer. The full RenderObjects feature is as shown :

Note that all objects are currently using the default sprite material, which places them into the Transparent render queue. We need to specify this in the Queue under the Filters on the feature.


Render Player Overlay

Then, we need to render the Player (again, since the Default Layer Mask includes that layer too), using another RenderObjects feature. This time testing against the value in the buffer, and overriding the material to show our overlay :


By using the Equal comparison function, we test for the value of 1 in the stencil buffer, which we wrote in the previous RenderObjects feature.

If the stencil buffer contains a value of 1, the test passes, so renders the pixel. If it fails, it discards the pixel.

We’ve also overridden the Material used for rendering, to use our SpriteOverlay material which is using the checkerboard Shader Graph example from earlier. This produces the final result as shown below and in the tweet at the start.


Thanks for reading!

If this post helped, consider sharing a link with others!

License / Usage Cookies & Privacy RSS Feed