Now that we have the basics of the VR sprayer simulation environment ready to go so we are getting to work on the program’s focus of operation and safety training. This is a critical week to incorporate features that allow for precise and meaningful teaching. To achieve this, we will be implementing multiple features including a tutorial system with a tutorial level, collision warnings, vital information, and various statistics to evaluate performance.
One thing that we knew we needed to create from the beginning was a tutorial for those entering both VR and a sprayer for the first time in order for them to get more familiar with quirks of VR experiences and the specific model sprayer. We decided to implement our tutorial by creating a reusable series of UI panels that appear and speak to the user, highlight objects, and direct the user on what the general steps of the sprayer operating process. We wrote a step-by-step script that first introduces users to VR. We have found over the years of working and demoing in the VR space that when a person who has never tried VR puts on the headset for the first time, a surprising percentage of them don’t even try to look around! It’s basically the main feature of VR, the freedom to look all around you, but people are so used to looking forward at a screen that we needed to address this in the beginning of the tutorial. Our tutorial first encourages the users to look around to find our Tutorial Spheres; this trains the user to both look around the environment as well as know they can look at a sphere if they need help continuing. We then guide the user through all the controls on the joystick and after that have them start the engines and drive the sprayer out of the barn.
Another major element we incorporated this week was collision detection on the sprayer. One of the primary goals with this training is to teach spatial awareness to operators so when operating the vehicle they don’t accidentally hit a telephone pole, tree, or other hazard with the boom arms extended out. With 120 foot boom arms, that’s a lot of area to be constantly aware of. Using box colliders on the boom arms and objects in the scene, if a collision is detected, we send an alert message to the sprayer display warning the operator of the collision and with what they collided.
Other warnings we are adding include the driver’s speed, which is important for the driver to control correctly, as it would affect the spray rate at which their product is distributed evenly on the crops. Weather and wind indication are also important to notify the user. Sprayer operators cannot do the application of product if the weather conditions are poor. If it’s too windy, product may end up in a neighboring field, or if it’s too wet, the sprayer could damage too much of the field or get stuck. We also added warnings for unsafe driving of the vehicle, such as having the arms extended in inappropriate areas such as the highway.
Finally, we added a top menu so the user can select different areas of the experience, as well as change settings and view their statistics. Currently this displays the number of objects the sprayer collided with as well as time operating and how much spraying was done. In the future, this will include more detailed information, such as per field spray coverage and any driving issues recorded.
Next week we will continue to improve the user experience in the application, as well as building out the first prototype hardware rig so we can do some early demoing of the simulator. Watch the video below for a more detailed look at this week's progress!
https://www.youtube.com/watch?v=ofDm8yui-0A