Driver Behavior Analytics Reference Implementation

Overview

The Driver Behavior Analytics Reference Implementation uses video injection and analytics and event recording services to monitor driver behavior. Develop the solution that provides alerts and stored video clips to drivers and fleet managers with this reference implementation.

Select Configure & Download to download the reference implementation and the software listed below.

Configure & Download

• Time to Complete: Approximately 60 minutes
• Programming Language: C++, Python*
• Available Software: Intel® Distribution of OpenVINO™ toolkit 2021.4.752 Release

Recommended Hardware

The below hardware is recommended for use with this reference implementation. See the Recommended Hardware page for other suggestions.

• ADLINK MXE-5500 Series

• NEXCOM VTC 7252-7C4IP

Target System Requirements

• Ubuntu* 20.04 LTS

• 6th to 10th Generation Intel® Core™ processors with Intel® Iris® Plus graphics or Intel® HD Graphics

How It Works

The application uses the inference engine included in the Intel® Distribution of OpenVINO™ toolkit and is designed to detect and track driver behavior and driver actions to ensure safe driving.

Using deep learning models, video streams, and analytics running on in-vehicle computers, it detects driver’s drowsiness states and distraction behaviors, and provides real time alerts to driver, and analytics per driver, truck and route over time.

Figure 1: Architecture Diagram

Get Started

Step 1: Install the Reference Implementation

Select Configure & Download to download the reference implementation and then follow the steps below to install it.

Configure & Download

NOTE: If the host system already has Docker* images and containers, you might encounter errors while building the reference implementation packages. If you do encounter errors, refer to the Troubleshooting section at the end of this document before starting the reference implementation installation.

1. Open a new terminal, go to the downloaded folder and unzip the downloaded RI package.

   unzip driver_behavior_analytics.zip
   

2. Go to the driver_behavior_analytics/ directory.

   cd driver_behavior_analytics/
   

3. Change permission of the executable edgesoftware file.

   chmod 755 edgesoftware
   

4. Run the command below to install the Reference Implementation.

   ./edgesoftware install
   

5. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download.

   

   Figure 2: Product Key

6. When the installation is complete, you see the message "Installation of package complete" and the installation status for each module.

   

   Figure 3: Installation Success

   NOTE: If you encounter any issues, please refer to the Troubleshooting section at the end of this document. Installation failure logs will be available at the path: /var/log/esb-cli/Driver_Behavior_Analytics_<version>/output.log

7. To start the application, change the directory using the cd command printed at the end of the installation process:

   cd /opt/intel/eif/EII-UseCaseManager/
   

Step 2: Run the Application

Prerequisites

• Set Up ThingsBoard* Cloud Data
• Set Up Amazon Web Services* Cloud Storage

1. Run the application:

   Copy and run the make webui command from the end of the installation log:

   make webui
   

2. Open the Web UI: Go to 127.0.0.1:9090 on your web browser.

   

   Figure 4: Reference Implementation Dashboard

3. If you installed your ThingsBoard Cloud Server and you have enabled S3 Bucket Server on your AWS account, you can provide your configured AWS Access Key ID, AWS Secret Access Key, AWS S3 Bucket Name Thingsboard IP, Thingsboard Port and Thingsboard Device token on the Cloud Data Configuration tab. After you complete the Cloud configuration, make sure you click on the Save Credentials and Save Token buttons.

   Now you can import the ThingsBoard dashboard as described at the end of the Set Up ThingsBoard* Cloud Data to enable all dashboard features, including the cloud storage.

   

   Figure 5: Configuration Tab Contents

   NOTE: If you don't have an AWS account, you will not be able to access Storage Cloud. You can still enable the Thingsboard Cloud Data if you configured it locally or on another machine.

4. In the sidebar, select the Driver menu option to set up a driver for the use case.

   Go to the video samples directory. /opt/intel/eif/EII-UseCaseManager/modules/EII-DriverBehavior-UseCase/config/VideoIngestion/test_videos/

   Open the video sample you will use to run the use case with your preferred video player. Make a screenshot profile picture of the driver's face.

   Access the Driver menu option on the dashboard. In Settings, add the profile picture and a driver name. Click on the Add Driver button.

   

   Figure 6: Driver Tab Contents

5. Access the Driver Behavior Analytics Dashboard with the following steps.

   • Go to sidebar and select Run Application.

     

     Figure 7: Select Driver Menu Option

   • Configure the use case by selecting the video sample and the device for all UDF models.

   • Optionally, you can also set the simulation data that you want to use. You can choose between using the KnowGo Simulator or simply use the CSV pre-recorded simulation data.

   Model Descriptions

   • Head Pose: Estimates the head(s) position in the video frame.

   • Facial Landmarks: Determines the facial landmarks of the identified people.

   • Face Detection: Detects the face(s) in the video frame.

   • Face Re-identification: Recognizes persons using the provided faces.

     

     Figure 8: Configure Use Case

   • Click on the Browse button and search for one of the sample videos delivered with the application at the following path: /opt/intel/eif/EII-UseCaseManager/modules/EII-DriverBehavior-UseCase/config/VideoIngestion/test_videos/ and select one of the two available.

     

     Figure 9: Select Sample Video

   • After selecting the video sample, select the device for all UDF models. Options include CPU, GPU, or HETERO:CPU, GPU. Click on Run Application.

   • The application will start the Visualizer App that will detect yawns, blinks, drowsiness, distraction status and will display the name of the driver as in the following image:

     

     Figure 10: Visualizer Output

6. After the visualizer starts, you can go to the ThingsBoard link and check the alerts sent by the reference implementation, the driver name and the profile picture previously added. If you configured the AWS credentials, you will also have access to video snapshots taken by the application on the video stream.

   

   Figure 11: Intel Fleet Manager Dashboard shown in ThingsBoard

7. You can also check the cloud storage from the Storage menu option.

   

   Figure 12: Reference Implementation Storage Tab

Run in Parallel with Vehicle Event Recording Reference Implementation

To run this task, you will need to download and install Vehicle Event Recording Reference Implementation.

For more details about parallel execution, see the Edge Insights for Fleet Use Case Manager documentation.

Prerequisites

• Follow the steps to install Driver Behavior Analytics after installing Vehicle Event Recording

Steps to Run the Application

1. Change directory to EII-Use Case Manager on your terminal:

   cd /opt/intel/eif/EII-UseCaseManager
   

2. Run the following command to start the web server application.

   make webui
   

3. Open your browser and go to 127.0.0.1:9090.

4. Configure both installed reference implementations by setting the video source and the target (CPU, GPU or HETERO). Click on Run Application.

   NOTE: Configure each reference implementation by selecting the desired tab. For example, click the Run Application menu option, then click on VER to configure the Vehicle Event Recording RI. Next, click on DBA to configure the Driver Behavior Analytics RI.

   

   Figure 13: Configure Vehicle Event Recording Reference Implementation

   

   Figure 14: Configure Driver Behavior Analytics Reference Implementation

5. Wait for both Visualizers to get up and running.

   

   Figure 15: Visualizer Output for 2 Reference Implementations

NOTE: If you reinstall the first reference implementation, you must reinstall the second reference implementation.

Summary and Next Steps

This application successfully implements Intel® Distribution of OpenVINO™ toolkit plugins for detecting and tracking driver behavior. It can be extended further to provide support for feed from network stream (RTSP camera), and the algorithm can be optimized for better performance.

As a next step, try the following:

Use deep learning models and driver facing camera video streams to detect driver’s drowsiness, distraction state and behaviors to provide real time alerts to driver. This reference implementation uses Intel® Distribution of OpenVINO™ toolkit Open Model Zoo pre-trained models and 3rd party models, but you can extend it to use your own models.

Learn More

To continue your learning, see the following guides and software resources:

• Intel® Distribution of OpenVINO™ toolkit documentation

Known Issues

Uninstall Reference Implementation

If you uninstall one of the reference implementations, you need to reinstall the other ones because the Docker images will be cleared.

Troubleshooting

Installation Failure

If the host system already has Docker images and its containers running, you will have issues during the RI installation. You must stop/force stop existing containers and images.

• To remove all stopped containers, dangling images, and unused networks:

  sudo docker system prune --volumes
  

• To stop Docker containers:

  sudo docker stop $(sudo docker ps -aq)
  

• To remove Docker containers:

  sudo docker rm $(sudo docker ps -aq)
  

• To remove all Docker images:

  sudo docker rmi -f $(sudo docker images -aq)
  

Docker Image Build Failure

If Docker image build on corporate network fails, follow the steps below.

1. Get DNS server using the command:

   nmcli dev show | grep 'IP4.DNS'
   

2. Configure Docker to use the server. Paste the line below in the /etc/docker/daemon.json file:

   { "dns": ["<dns-server-from-above-command>"]}
   

3. Restart Docker:

   sudo systemctl daemon-reload && sudo systemctl restart docker
   

Installation Failure Due to Ubuntu Timezone Setting

While building the reference implementation, if you see /etc/timezone && apt-get install -y tzdata && ln -sf /usr/share/zoneinfo/${HOST_TIME_ZONE} /etc/localtime && dpkg-reconfigure -f noninteractive tzdata' returned a non-zero code: 1 make: *** [config] Error 1

Run the following command in your terminal:

sudo timedatectl set-local-rtc 0

Installation Encoding Issue

While building the reference implementation, if you see ERROR: 'latin-1' codec can't encode character '\\u2615' in position 3: ordinal not in range(256)

Run the following command in your terminal:

export LANG=en_US.UTF-8

Can't Connect to Docker Daemon

If you can't connect to docker daemon at http+docker://localhost, run the following command in your terminal:

sudo usermod -aG docker $USER

Log out and log back in to Ubuntu.

Check before retrying to install if group Docker is available for you by running the following command in a terminal:

groups

The output should contain "docker".

Installation Timeout When Using pip or apt Commands

You may experience a timeout issue when using the People's Republic of China (PRC) internet network.

Make sure that you have a stable internet connection while installing the packages. If you experience timeouts due to Linux* apt or Python* pip installation, try to reinstall the package.

Support Forum

If you're unable to resolve your issues, contact the Support Forum.