My Superman Vision: the Power of a Shader

Hi, welcome to the new post of Flying Turtles Team for the Intel Ultimate Coder Challenge. In this post we will see some more technical and advanced aspects regarding shaders.

An important aspect and a great advantage of training with Mixed Reality is the possibility to see objects from impossible perspectives in a normal training session.

One of my favorite TV shows is "How is made" because it allows you to understand how many common objects are made inside.

Knowing how complex industrial machinery are made inside them is very important, but also very difficult to explore with a real machine. For this reason, usually the technical staff in charge of repairs or maintenance workman studies the procedures on 2D paper manuals or in some cases digital, but never sees the inside of the machine until they have to perform a real maintenance operation.

In our application TraIND4.0 we have decided to give the possibility to the users to slice with some cutting plans the objects in the scene and to be observe the inside in a different way from the classic explosion of components that is seen in many VR applications.

To achieve this effect we have created a new shader dedicated to this feature.

Often surrounded by mystery, a shader is a program specifically made to run on a GPU. It is, ultimately, what draws the triangles of any 3D models. Shaders are essential to give a special look to your application. Unity also uses them for postprocessing, making them essential for 2D games as well.

MixedRealityToolkit has introduced with the release of May a new “Standard” shader optimized for MixedReality. One of the interesting aspects of this shader is the possibility to have a cutting plane called "ClipPlane" to generate edge effects on objects or to manage in a nicer and more elegant way the view when the camera is too close to objects. The StandardShader can be found in the MixedRealityToolkit or at this link: https://github.com/Microsoft/MixedRealityToolkit-Unity/blob/master/Assets/HoloToolkit/Common/Shaders/Standard.shader

Looking at how it is done internally, the part of our interest is as follows:

#if defined(_CLIPPING_PLANE)
            inline float PointVsPlane(float3 worldPosition, float4 plane)
            {
                float3 planePosition = plane.xyz * plane.w;
                return dot(worldPosition - planePosition, plane.xyz);
            }
#endif
                // Plane clipping.
#if defined(_CLIPPING_PLANE)
                float planeDistance = PointVsPlane(i.worldPosition.xyz, _ClipPlane);
#if defined(_CLIPPING_PLANE_BORDER)
                fixed3 planeBorderColor = lerp(_ClippingPlaneBorderColor, fixed3(0.0, 0.0, 0.0), planeDistance / _ClippingPlaneBorderWidth);
                albedo.rgb += step(planeDistance, _ClippingPlaneBorderWidth) * planeBorderColor;
#endif
#if defined(_ALPHA_CLIP)
                albedo *= step(0.0, planeDistance);

#else
                albedo *= saturate(planeDistance);
#endif
#endif

In this piece of code, for each pixel is calculated the distance between the object and the Clipping Plane and consequently assigned or not the texture / color.

Starting from this we have extended the shader to be able to have more plans and to be able to manage nested materials.

#if defined(_CLIPPING_PLANE)
                float planeDistance = PointVsPlane(i.worldPosition.xyz, _ClipPlane);
				float planeDistance2;
				if (_ClipPlane2Enabled>0.0) 
					planeDistance2 = PointVsPlane(i.worldPosition.xyz, _ClipPlane2);
#if defined(_CLIPPING_PLANE_BORDER)
                fixed3 planeBorderColor = lerp(_ClippingPlaneBorderColor, fixed3(0.0, 0.0, 0.0), planeDistance / _ClippingPlaneBorderWidth);
                albedo.rgb += step(planeDistance, _ClippingPlaneBorderWidth) * planeBorderColor;
#endif
#if defined(_ALPHA_CLIP)
                //LOGICA CLIP PLANE				
	if (_ClipPlane2Enabled > 0.0) {
		if (_ClipPlaneCutMode > 0.0) {
			albedo *= step(0.0, planeDistance) || step(0.0, planeDistance2);//||
		}
		else {
			albedo *= step(0.0, planeDistance) && step(0.0, planeDistance2);//&&
			}										
	}
	else {
		albedo *= step(0.0, planeDistance);
	}
#else
                albedo *= saturate(planeDistance);
#endif
#endif

The new shader calculates the distance from 2 floors with different combination options that you can see in the video.

The complete shader will be published directly on the MixedReality Toolkit.