See Through Objects With Stencil Buffers Using Unity URP

Overview

In this blog post, we’ll look at a simple way to create objects that can be masked so that they disappear using stencil buffers inside Unity’s Universal Render Pipeline (URP). URP is Unity’s premade Scriptable Render Pipeline (SRP) and is created to support as many platforms as possible. URP trades graphics quality for performance and attempts to increase performance for games on less powerful devices, such as mobile. We’ll start by setting up the project. Then, install a custom shader we’ll need. Finally, we’ll put it all together using the Render Objects feature of the URP’s forward renderer.

Setting Up The URP

The rendering pipelines in Unity have gone through a lot of changes in the recent years. We’ll be using the latest Unity version 2019.3 with the Universal Render Pipeline package version 7.1.8. URP was the Light Weight Render Pipeline (LWRP) in earlier versions of Unity, so if you see mention of that, you probably just need to update your Unity version.

If you are running a version of Unity 2019.3, you can install the URP right from the project creation menu by clicking on the URP Template when naming your project. If you don’t want all of the assets that the URP template imports, you can create a blank 3D project in Unity and then do the following:

• Go to Window -> Package Manager.
• Search for ‘Universal RP’ in All packages. Click Install at the bottom right.
• In your project window, click the +, or right click and select Create -> Rendering -> Universal Render Pipelines -> Pipeline Asset (Forward Renderer)
• Go to Edit -> Project Settings -> Graphics and under ‘Scriptable Render Pipeline Settings’ click on the circle next to ‘None’ and select your URP asset you just created.
• Now you are using the Universal Render Pipeline!

Note: If you are seeing pink objects, you may need to upgrade your existing shaders by going to Edit -> Render Pipeline -> Universal Render Pipeline -> Upgrade Project Materials to UniversalRP Material.

Setting Up Shaders & Materials For Stencil Buffers

As of this writing, Unity’s Shader Graph tool doesn’t allow users to set Stencil Buffers on a shader. We can get around this by creating a very simple shader and then configuring the stencil buffer options using the URP’s Forward Renderer. First, we’ll create a new shader. In your project panel, go to Create -> Shader -> Unlit Shader. Name it mask and open it up. Replace all of the contents with the following:

Shader "Custom/Mask"
{
	Properties{}

	SubShader{

		Tags {
			"RenderType" = "Opaque"
		}

		Pass {
			ZWrite Off
		}
	}
}

We set the RenderType to Opaque so that our Mask will be rendered with other Opaque shaders. The ZWrite Off tells the renderer to not render any pixels, so we won’t see our mask. Right click on the shader and click Create -> Material. Now, we’re ready to set up our scene to test.

Scene Setup

Do the following:

• Create a 3D Cube at 0, 0, 3.
• Create a 3D Capsule at 0, 0, 0 and name it ‘SeeThrough’.
• Click to change its Layer and click Add Layer… Then add two new Layers Mask and SeeThrough
• Assign the Capsule to the ‘SeeThrough’ layer.
• Create a 3D sphere at 0, 0, -1 and name it ‘Mask’.
• Assign it the Mask layer.
• Under its Mesh Renderer change the material from ‘Lit’ to your mask material.
• Change Cast Shadows to ‘Off’

Pretty quick setup. Once completed, you should be able to see through the mask sphere to the capsule behind it. But not the cube, yet. Now, let’s configure our forward renderer!

Stencil Buffers With URP Forward Renderer

Next up, find your Forward Renderer inside of your project panel and click on it. If you didn’t change the name, it should be ‘UniversalRenderPipelineAsset_Renderer.’ Settings should pop up in the inspector.

Note: There may be a bug that prevents your changes from saving. Specifically on the Renderer Features items. If you add the renderer features, then save your scene, you should see them pop up in the Project panel nested under your Renderer. Clicking on these individually should let you update the values. If it really isn’t updating, delete it and re-add.

• Under Default Layer Mask, uncheck SeeThrough and Mask. This prevents the renderer from rendering these two layers.
• Next, we’ll add them back in by clicking the + under ‘Renderer Features’ and adding Render Objects (Experimental).
• Name this render feature Mask.
• Change its Event to Before Rendering Opaques.
• Change its Layer Mask to Mask. This means that we’ll add the stencil overrides only to the ‘Mask’ layer.
• Next, check the Stencil option. Set its Value to 1, Compare to Always, and Pass to Replace. (You may need to click on the Mask item nested under your Renderer in your project panel for these settings to save. See the bug above.)
• Add another Renderer Feature by clicking the + on our Forward Renderer.
• Name it SeeThrough.
• Change its Event to Before Rendering Opaques and Layer Mask to SeeThrough.
• Now, check the Stencil option. Set its Value to 1 and Compare to NotEqual.

At this point, if you click on your game camera view, you should see through the capsule object to the cube behind it. In the scene view, you can rotate your view and move your sphere to make this more apparent. The completed settings should look similar to the following:

‍

Parameters For Stencil Buffers

So, what is happening? First, we took our Mask and SeeThrough layers out of the Forward Renderer completely. At this point, without adding any Renderer Features, they should disappear completely from view. Immediately after that, we added in two Render Objects features and assigned one to the Mask layer and another to the SeeThrough layer at the Before Rendering Opaques event. This told our Renderer to add back in those two layers to rendering before other Opaques shaders are rendered. Finally, we overrode the stencil buffer options for those two entire layers.

For the Mask layer, using a compare function of Always means that the Pass operation is always run. Setting the Pass to Replace means that all of the pixels inside of our masks mesh will have a stencil value set to 1. For the SeeThrough layer, setting the compare function to NotEqual means that all of the pixels on layer SeeThrough with a Stencil value not equal to 1 will be rendered. Because the pixels behind our mask were set to 1 in the earlier step, this means that everything behind our mask sphere will not be rendered.

Notes

The ordering is important here. If you put the SeeThrough Render Objects Feature above the Mask, the Mask will set the value to 1 after the SeeThrough layer does its check, and nothing will happen.

Also worth noting, if you are using Cascading Shadows in you URP renderer, currently there seems to be a glitch that they render above your SeeThrough layer. Turning Cascades off should render as normal.

You might be confused why for the SeeThrough’s Pass and Fail are Keep still. Keep simply means that the buffer value doesn’t change, it remains 1. The compare function does two things. First, it only renders the pixels that match its condition using the reference value. Second, based on whether the condition comes back true or false, the pass or fail operations are run. This gives the option to change the reference value again. For more information, check out the Unity docs on stencil buffers.

Conclusion

To conclude, we dove into the Universal Render Pipelines custom features and created a layer that we can x-ray through with our mask objects. There are a lot more cool features we can do with the Render Objects feature. We could use it to show outlines of units hiding behind buildings, or use it to apply custom materials to objects at certain points in our render. It’s unclear how Shader Graph will develop. But it’s possible that in the future, Unity will allow us to add Stencil Buffers directly to shaders, without needing to override them on specific layers.

‍