The success of games such as Unreal Tournament* and Robo Recall* have a lot to do with how the lighting is handled in Unreal Engine*. One reason for this is static light-mapping, but it does come at a price: building lights demand time, and time is money. This article will share the long process of improving our production pipeline, with a focus on tapping into the full power of all our office Intel® processors.
As the name suggests the primary focus of VRMonkey is virtual reality (VR). VR is a challenge since not only are most of the technologies new, but the computational demands are usually higher than what most company PC’s can handle. Struggling to find solutions that are both economical and fast to integrate into a work environment led VRMonkey to choose Unreal Engine* 4 for designing high-quality VR experiences.
As a free-to-use and open source solution, Unreal Engine offers robust tools that are frequently unused, either due to the complexity of their configurations or simply due to a lack of knowledge about these advanced options. One of its most significant advantages is the ability to define lights as “Static Lights” and to print these into textures which define the global illuminations of the virtual world. This feature makes it possible to avoid real-time processing of the lights while also saving precious resources that can instead handle more polygons or effects on the screen.
One of VRMonkey’s projects consisted of 13 different VR scenes, called maps in Unreal, and each of the scenes needed the lighting built. For each map, building the light would take seven to ten hours; and most of the time we could only tell if the changes were good or not by repeating the process, over and over again.
At that time, the artists doing the light building were using either an Intel® Core™ i7-4710HQ or Intel® Core™ i5-4460 processor. To be clear, this light building process is only CPU bound. After reading about using a swarm of PCs to handle this process, VRMonkey got an Intel® NUC Kit NUC6i7KYK (Skull Canyon) with a 6th generation Intel® Core™ i7-6770HQ processor. This low-power machine could be kept on at all times in the office. With this capability, VRMonkey managed to speed up the building time to less than an hour.
Dinos do Brasil
When VRMonkey founders, Pedro and Keila Kayatt, got their hands on the first Oculus Rift* Developer Kit (DK1) they started to think about what would be a great way to use such amazing technology. They first discussed the possibility of creating a virtual trip inside the body, just like in the 1987 movie Innerspace, and then recalled a project from their childhood where a giant doll, the Eva Doll, was created that allowed people to walk inside and learn how the organs worked.
That was a great idea, but then they got another bigger, crazier one – to re-create the world of dinosaurs, and taking that a step further, show the world that Brazil (which is one of the largest countries in the world) had dinosaurs. This was how the idea for the Dinos do Brazil project was born in 2013.
Figure 2. Dinos do Brasil Illustration
They began by assembling a team of researches and professionals that could bring to life something that was scientifically accurate as well as amazingly beautiful. They created a dream team for a dream project by bringing together paleontologists, paleoartists, 3D designers, paleobotanics, programmers and so many talented people. Pedro and Keila began looking for investments to fund the project, eventually receiving approval from Rouanet Law, a law that trades taxes for investments in cultural projects. They immediately started working on a prototype to reach companies.
By then it was 2015, and a small team worked for six months to create a prototype, taking that to companies to try to raise funding. The crisis in Brazil was not helping them, but they did have a lot of contacts from years of partnership with several companies, Intel included. VRMonkey showed this project to them, and by the very end of the year, they finally got funded. They reached the minimum quota with a final investment from Intel Brasil, which totaled an amount of around US$200,000 from the total of $1 million that they understood would be ideal to build the project. It was not going to be easy, but hardly any great things are.
After several meetings and scripts, the format was decided – 13 scenes ranging from space to different environments around the area representing present-day Brazil showing 15 different pre-historic creatures. All of it would be virtually modeled, animated, and assembled in Unreal Engine 4. Over 20 people worked on the project, and VRMonkey soon found themselves facing a very common problem regarding the development of VR experiences: performance.
They needed to reach a framerate of 90 frames per second (fps) and to be able to do that with a not-so-expensive rig. Why? Well, the project had a major turn of events. After looking for the best partners, VRMonkey hit the jackpot when talking with the amazing people from Catavento Cultural, one of the largest museums in Latin America. Catavento Cultural fell in love with the project and offered to place the project as a permanent exhibition, making it possible for children from all around Brazil to visit. The exhibit would consist of 25 chairs, each one hosting a VR full-setup, including PC and an Oculus Rift* CV1 VR Headset. All of this was amazing, but with one problem: the cost of all the equipment would be charged to VRMonkey. Long story short, about half of the investment money was about to be spent on hardware.
Figure 3. Dinos do Brasil permanent VR exhibit at Catavento Cultural
Getting back to the performance matter, how could VRMonkey make a realistic render in VR without sacrificing precious frames? Their first idea was baking lights. Lights can be approached in two main ways: a dynamic light that changes how the shadows are processed by the game engine in runtime or a static light, which is a light that cannot be changed when running the experience. Unreal Engine is known for how well they handle static lights, creating very realistic bounces of light and making it possible to create a nice illumination over the environment by building the lights over the object textures.
But static lights have a downside; they must be processed. This means they need to be compiled by a lighting engine and this takes a lot of time to be done in the right way. Unreal Engine even has a pre-set of configurations to make it easier to preview how the lights will turn out, but even in the preview mode some of the scenes would take between seven to ten hours to render. With 13 scenes, it could take up to 130 hours to have the preview of the lights for the project, which was worrisome.
It was about that time that Pedro returned from the Intel® Innovator Summit with an Intel® NUC Kit NUC6i7KYK (Skull Canyon), a real beast in a small case, holding a 6th generation Intel® Core™ i7-6770HQ processor with 16GB RAM and 256GB SSD. This machine alone was already doing amazing VR work, matching several of the desktop stations at VRMonkey.
Then Pedro remembered reading about the swarm application for Unreal Engine. Basically, connect several computers in a local network and make them available to the entire set of cores in the processor. For instance, Pedro configured the Intel NUC to share all eight of its cores to the swarm, and the other working stations to share 3 or 4, creating less impact on the people working on those machines.
Figure 4. Networking provided a lot of computational power to be available
VRMonkey connected all of the local networks with CAT5 Ethernet cables, allowing for the transfer of files into LAN at 100MB/s – a fantastic rate. They were also able to run the entire swarm with Intel NUC as the coordinator, speeding up the light processing by more than ten times what it was previously. They could essentially re-run the lights from all the scene in less than a day with this setup.
After a year of hard work, Dinos do Brasil was complete - launching in March of 2017. As of this writing, the exhibit has had over 100 thousand visitors, and more and more people are experiencing VR for the first time there every day.
Pedro and Keila’s dreams and efforts were paid back in full through the reactions they see in kids who probably would have to wait another five or ten years to experience such amazing technology and in the tears of the paleontologists who saw their discoveries come alive.
How to Configure a Light Swarm
You might be thinking that setting up a Swarm is be a complicated task, but in fact, it is not that hard. You just to need to understand a few crucial points before starting, and before you know it, you will have it up and running. First, select which computer will be the coordinator; it will need to be on most of the time and will handle all the other PC connections.
The steps below use an Intel® NUC Kit NUC6i7KYK (Skull Canyon) as the coordinator since it is a low Thermal Design Power (TDP) machine and can run 24/7 without impact.
Once the Intel NUC is set up, go to the folder:
There you will find the following files:
Configure the Coordinator
These files are independent and can be copied to your coordinator in any folder. Let’s put all them inside a folder called Swarm and drop it on the desktop of the selected machine you’ve selected to be the coordinator. When you open the SwarmCoordinator.exe, you should see this:
One key thing to note: This application must always be open for the other peers to connect. With that in mind, the next step is to make sure it is always open when turning on the PC. The easiest way to do this is to execute the Run command from Windows* (Win+R) and type “shell:startup”, this will open a Windows Explorer at the address: C:\Users\[YOURUSER]\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup
Drag the SwarmCoordinator.exe by holding the Alt key, and it will create a shortcut for it. Now, every time you start your computer, the Coordinator executable program will automatically start.
Set up the Coordinator to be a Peer
The coordinator can also be set up to be a peer; this means that the computer that handles the connections can also share its processor power with your Swarm. To do this, setup the Swarm Agent by opening the SwarmAgent.exe file.
Figure 6. The SwarmAgent.exe file will open in your task bar as shown here
You can also add a shortcut of this executable to the Windows Startup. Do not do this though if this computer can have its performance undermined, since light compilations take full use of the CPU. The next step shows how to configure the agent in a way that you can specify how many cores it will take from the CPU.
Configure the Swarm Agent
Double click on the icon to open the Swarm interface, this Agent will open automatically when you are building the light from Unreal*, but it will always open from your UE4ROOT (Installation folder from Unreal). If you have a Swarm file previously open (the way you do now using the Startup Folder) it will only direct the calls from the engine to this version of the application. This is important since some people close the Swarm (manually) causing Unreal to automatically opens a new one that won’t have any of your configurations in it.
Settings Tab Configurations
Several fields that must be changed on the Settings tab.
CoordinatorRemotingHost must point to the IP Address (usually something like 192.168.1.X) of your Coordinator machine. As an option, it can be the machine name of it (as shown in Figure 7, skullcanyon is used). Note: Just after changing this option the app will freeze since it will be connecting to the coordinator. By using a different window, you can see that the new agent has connected to the coordinator, as shown in Figure 8.
As you can see in Figure 7 (in the blue square), both AgentGroupName and AllowedRemoteAgent Group are the same, this is simply because I wanted my machine to deploy the light build and also to do the light build for the other machines.
AllowedRemoteAgentNames change from the default of RENDER* to just *. This setting will allow any machine on the network, with the swarm configuration, to be able to build the lights.
ShowDeveloperMenu change this to true as this will allow you to make the rest of the configurations that are needed.
DeveloperSettings Tab Configuration
Now that the SwarmAgent is configured, you may want to change some DeveloperSettings.
On this tab, as shown in Figure 9, you need to set the number of processors (logical ones) that should be used and the priority of the jobs depending on a Local Light Build or a Remote one (requested by another machine). Keep in mind that if the CPU is already BUSY the Swarm Agent will inform the coordinator and no jobs will be delivered to your Agent.
Configure Peer Machines
Set the Swarm Agents to run on each one of your peer machines (client machines). To do this, copy the same files from the folder of the Swarm Coordinator ([UE4ROOT]\Engine\Binaries\DotNET), but you now have the ability to place it in a more discrete folder, such as C:/UnrealSwarm/ (or anything you prefer). Remember to set the SwarmAgent.exe to have a shortcut to the Startup Folder, but don’t do it for the SwarmCoordinator.exe.
Open the SwarmAgent.exe from this folder and set all the configurations that we have applied above for the coordinator. Then, you can customize the local and remote processor count in the DevSettings tab as you wish. Do this for each PC on your network, and before you know it you will have a Light Swarm ready to use for your Unreal projects!
For information, refer to the Unreal Swarm* Documentation Overview.
Integrating Newer Processor Technology into the Setup
With everything running great on 6th generation technology, what would happen if you upgrade to use the power of the 8TH generation Intel® Core™ i7 processor? VRMonkey has been considering a more powerful PC to run photorealistic VR experiences – so they decided to give it a try.
VRMonkey settled on an Alienware* Aurora* desktop, a mid-tower monster with an Intel® Core™ i7 8700K processor, 32GB RAM with Dual Channel HyperX* DDR4 at 2933MHz and an NVIDIA GeForce* GTX 1080 with 8GB GDDR5X.
Figure 10. The beautiful new addition to VRMonkey offices
This new generation of processor increases the number of physical cores from 4 to 6 - powering up from 8 to 12 logical processors running at 3.7GHz! VRMonkey quickly set it up on their Swarm to test it out.
VRMonkey got their system up and running so smoothly, that for the particular scene shown in Figure 11 it actually took longer to export the scene than to compile process the data back. Pedro found an article comparing CPU performance in the Unreal Editor for Unreal Engine 4.
The 8th generation Intel Core i7 8700K processor was not included in the comparison, so Pedro used a similar approach to start comparing the building times of the different setups they do use at the VRMonkey office. He was able to generate the graph below which shows how much time a default scene on Unreal takes to be built individually.
The VRMonkey team sees the value that a powerful processor can have on not only individual machines but how much it can positively impact the performance of the swarm.
In addition to improving these pipelines, VR Monkey has a new project to make changes to the Source Code of Unreal Engine. They want to use the same strategy they used with the Static Light builds and apply that to the Unreal Source Code. With this in mind, they are looking for solutions, one possibility that appears to be almost ready for Unreal Engine is IncrediBuild*. It’s a free application, but has a lot of limitations. As they continue looking, another promising solution could be Fastbuild. Keep an eye out for updates on what options they decide to pursue as well as other tips (such as Shared Data Cache) in future articles exclusively here at Intel® Developer Zone.