One of the greatest parts of this project is the fact that Team Crop Dusters were able to work with an Intel Hades Canyon NUC to be the computer running the simulator. The Hades Canyon is not much bigger than the size of a book and its size is perfect for keeping our mobile unit at a low profile and portable. Despite its small footprint there was not a lot of optimization that needed to be done specifically for the device, but there were a few things we needed to do in general due to the scope of the project.
The main slowdown on any system we ran the simulation on was the number of vertices being displayed on screen at one time. The more vertices that need to be displayed, the slower the system ran and the worse the experience became. One way to curtail this is to perform vertex reduction on the various models in the scene. Our sprayer model is the main culprit here, coming in at around 1.3 million points. Simply placing the model in the scene was not practical in terms of getting reasonable frames per second. To avoid the potential motion sickness, the model had to be decimated and removed where parts weren’t being actively viewed.
Occlusion culling is the process in which objects that are occluded (blocked from vision by another object that is closer) are not rendered. If enough objects are not rendered in a scene when playing, the frame rate will get a definitive bump. Frustum Culling is the idea of taking only the slice of what you see and not rendering the rest. You can think of this one as a pie chart with only 1/3 of it filled. If you stand in the center and look in the direction of the third then you only need to render that part!
Unity provides another feature for optimization called Level of Detail that we decided to take advantage of. As the user moves further from an object, less detail is actually needed to make an object look the same as when the same object is viewed closer, thus fewer vertices can be used. Unity calls its implementation of this concept, Level of Detail Groups or (LOD Groups). By adding the component to a Gameobject and assigning other models to show at certain distances fewer vertices will need to be rendered and fps should increase.
Utilizing LODs was very useful when rendering farm equipment and objects, so we also tried to utilize it with our corn models. Originally, we used 3D models to render the corn. Unfortunately, with 3D models, sometimes the only way to get detail is by increasing your vertices count. We attempted to utilize LODs on the corn models, but even this was not enough of an improvement. To fix this, we switched the corn model to 2D sprites. We utilized an old-school technique to make the corn sprites look 3D by having two sprites layered on top of each other, with one rotated at a bias. This makes the sprite look fuller. Doing this we were able to enhance the density of the corn fields and show visible corn rows without taking a hit to our performance.
Corn Sprite Layering
To further optimize both the experience and frame-rate we reduced the overall size of the farmland terrain to approximately 1/4 of its original size. This improved the overall speed of the simulation and let the operators focus more on what they have to work with.
One thing that was slowing the simulation down was the presence of three mirrors (additional cameras) located on the sprayer boom arms that also had to render the scene. This was slowing it down enough to cause some motion sickness while viewing. The way we decided to address this issue was by only having one mirror active in the scene at a time while you looked the relevant one and disabling the rest. For this it was Dot Product to the rescue. The dot product formula finds the portion of one vector that lies on another vector and returns one number. Here, a light shining on an angle represents it:
We took the dot product of the vectors from the camera to the mirrors and if it was greater than a viewAngle variable we turned on that mirror and turned off the rest. It worked like a charm and the frame rate improved once again.
Optimization can be a lot of work, but with these things in place we can improve the quality of the simulator while maintaining high frame rates. Low frame rates lead to motion sickness with VR, which we need to avoid. For most of our users, this will be their first VR experience. The important thing is that our users are focused on the training experience and not the technology running it.
Watch this week's video below to see how we optimized the VR sprayer simulation:
https://www.youtube.com/watch?v=2zkwQbau0JM