Automated License Plate Recognition Reference Implementation

Version: 2022.1   Published: 12/09/2021  

Last Updated: 06/28/2022

Overview

Automated License Plate Recognition demonstrates how to use Edge Insights for Fleet middleware and delivers deep learning models, computer vision algorithms, OpenVINO™ and other software. In this example, the model outputs detected license plate numbers on a dashboard. This could be used in scenarios where warehouse managers need to know which vehicles have arrived or left the warehouse. Front facing dash camera video streams are analyzed to extract the license plate text strings. The results are transmitted and made available via an in-vehicle local dashboard and a cloud dashboard. The application also temporarily stores relevant video images for validating the accuracy of detections.

Select Configure & Download to download the reference implementation and the software listed below.

Configure & Download

Legal Disclaimers
Recipient is solely responsible for compliance with all applicable regulatory standards and safety, privacy, and security related requirements concerning Recipient's use of the Intel hardware and software.

Recipient is solely responsible for any and all integration tasks, functions, and performance in connection with use of the Intel hardware or software as part of a larger system. Intel does not have sufficient knowledge of any adjoining, connecting, or component parts used with or possibly impacted by the Intel hardware or software or information about operating conditions or operating environments in which the Intel hardware or software may be used by Recipient. Intel bears no responsibility, liability, or fault for any integration issues associated with the inclusion of the Intel hardware or software into a system. It is Recipient’s responsibility to design, manage, and assure safeguards to anticipate, monitor, and control component, system, quality, and or safety failures.


  • Time to Complete: Approximately 60 minutes
  • Programming Language: Python*
  • Available Software: Intel® Distribution of OpenVINO™ toolkit 2021.4.2 Release

Recommended Hardware

The hardware below is recommended for use with this reference implementation. See the Recommended Hardware page for other suggestions. 


Target System Requirements 

  • Ubuntu* 20.04.4 LTS

  • 6th to 10th Generation Intel® Core™ processors with Intel® Iris® Plus graphics or Intel® HD Graphics


How It Works

The reference implementation contains a full pipeline of analytics on video streams from a front facing dash camera on an Intel® Core™ or Intel Atom® processor-based computer onboard the fleet vehicle. Pretrained models are used to inference and extract the license plate information.

This reference implementation contains a notification subsystem which includes a local dashboard, and a cloud dashboard. 

Architecture Diagram


Get Started

Step 1: Install the Reference Implementation

Select Configure & Download to download the reference implementation and then follow the steps below to install it.

NOTE: The images provided in the reference implementation are ONLY to be used for validating the accuracy of detection events.

Configure & Download

NOTE: If the host system already has Docker* images and containers, you might encounter errors while building the reference implementation packages. If you do encounter errors, refer to the Troubleshooting section at the end of this document before starting the reference implementation installation.

  1. Open a new terminal, go to the downloaded folder and unzip the downloaded RI package. 
    unzip automated_license_plate_recognition.zip
  2. Go to the automatedlicenseplate_recognition/ directory.
    cd automated_license_plate_recognition/
  3. Change permission of the executable edgesoftware file.
    chmod 755 edgesoftware
  4. Run the command below to install the Reference Implementation.
    ./edgesoftware install
  5. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download.
    Screenshot of Product Key

  1. When the installation is complete, you see the message "Installation of package complete" and the installation status for each module.
    Screenshot of Installation Complete

NOTE: If you encounter any issues, please refer to the Troubleshooting section at the end of this document.
Installation failure logs will be available at the path: /var/log/esb-cli/Automated_License_Plate_Recognition_2022.1/output.log

  1. In order to start the application, you need to change the directory using the cd command printed at the end of the installation process:
    cd <INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase

Step 2: Run the Application

Prerequisites

  1. Run the application. 

    Copy and run the make webui command from the end of the installation log:

    make webui EII_BASE=<INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_<version>/IEdgeInsights REPO_FOLDER=<INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_<version>/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase

    For example: 

    make webui EII_BASE=/home/intel/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/IEdgeInsights REPO_FOLDER=/home/intel/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase
  2. Open the Web UI: Go to 127.0.0.1:9095 on your web browser. 

    Screenshot of ThingsBoard Web GUI

  1. If you installed your Thingsboard Cloud Server and you have enabled S3 Bucket Server on your AWS account, you can provide your configured AWS Access Key ID, AWS Secret Access Key, Thingsboard IP, Thingsboard Port and Thingsboard Device token on Cloud Data Configuration tab. After you complete the Cloud configuration, make sure you click on the Save Credentials and Save Token buttons. Now you can import the ThingsBoard dashboard as described at the end of the Set Up ThingsBoard* Cloud Data to enable all dashboard features, including the cloud storage. 

    Screenshot of AWS Configuration

NOTE: If you don't have an AWS account, you will not be able to access Storage Cloud. You can still enable the ThingsBoard Cloud Data if you configured it locally or on another machine.

  1. Access the Automated License Plate Recognition Dashboard with the following steps. 
  • Go to sidebar and select Run Use Case.
    Screenshot of Run Use Case

  • Configure the use case by selecting the video sample and the device for all UDF models.

    NOTE: These images are ONLY to be used for validating the accuracy of detection events.

    Model Description
    Both models below are enabled by default.
    Licence Plate Detection: This model detects (Chinese) vehicles licence plates.
    Licence Plate Recognition: This model recognizes Chinese license plates in traffic.



    Screenshot of Dashboard

  • Click on the Browse button and search for converted video prepared in Prerequisites
  • After selecting the video sample, select the device for all UDF models. Options include CPU, GPU, or HETERO:CPU,GPU. Click on Run Use Case

    NOTE: To use a GPU or a HETERO:CPU,GPU combination device, you must set the proper group for the device with the command:

    sudo chown root:video /dev/dri/renderD128

     

  • The application will start the Visualizer App that will detect license plates and license plate numbers as shown in the following image:
    NOTE: These images are ONLY to be used for validating the accuracy of detection events.
    Screenshot of Visualizer

  1. After the visualiser is started, you can go to the ThingsBoard link and check the alerts sent by the reference implementation. If you configured the AWS credentials, you will also have access to video snapshots taken by the application on the video stream. 
    Screenshot of ThingsBoard Dashboard with Data

  1. You can also check the cloud storage from the Reference Implementation Storage tab.
    NOTE: These images are ONLY to be used for validating the accuracy of detection events.
    Screenshot of AWS Storage


Run in Parallel with Public Transit Analytics Reference Implementation

To run this task you will need to download and install Public Transit Analytics Reference Implementation.

Prerequisites

Steps to Run the Application

  1. Change directory to Public Transit Analytics Use Case path on terminal 1:
    cd <INSTALL_PATH>/public_transit_analytics/Public_Transit_Analytics_2022.1/Public_Transit_Analytics/EII-PassengerCounting-UseCase


    Screenshot of Change Directory

  2. Change directory to Automated License Plate Recognition Use Case path on terminal 2:
    cd <INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase
    Screenshot of Change Directory ALPR

  1. Run the following command on terminal 1 to start the webserver application: 
    Copy and run the make webui command from the end of the installation log:
    make webui EII_BASE=<INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/IEdgeInsights REPO_FOLDER=<INSTALL_PATH>/public_transit_analytics/Public_Transit_Analytics_2022.1/Public_Transit_Analytics/EII-PassengerCounting-UseCase

    For example: 

    make webui EII_BASE=/home/intel/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/IEdgeInsights REPO_FOLDER=/home/intel/public_transit_analytics/Public_Transit_Analytics_2022.1/Public_Transit_Analytics/EII-PassengerCounting-UseCase

     

  2. Run the following command on terminal 2 to start the webserver application:
    Copy and run the make webui command from the end of the installation log:
    make webui EII_BASE=<INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/IEdgeInsights REPO_FOLDER=<INSTALL_PATH>/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase

    For example: 
    make webui EII_BASE=/home/intel/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/IEdgeInsights REPO_FOLDER=/home/intel/automated_license_plate_recognition/Automated_License_Plate_Recognition_2022.1/Automated_License_Plate_Recognition/EII-LicensePlateRecognition-UseCase

    Screenshot of Web Server Application
  3. Open your browser and go to 127.0.0.1:9094

  4. Configure Public Transit Analytics by setting the video source and the UDF target. Next, click on Run Use Case.

  5. Wait for Visualizer to get up and running.

  6. Open the Automated License Plate Recognition page by going to address 127.0.0.1:9095.

  7. Configure all available cameras with the desired videos. Set the target for each UDF. Options include CPU, GPU, or HETERO:CPU,GPU. Click Run Use Case.
    NOTE: These images are ONLY to be used for validating the accuracy of detection events.
    Screenshot of Configure ALPR

At this point Public Transit Analytics will close and after that both use cases will start.
NOTE: These images are ONLY to be used for validating the accuracy of detection events.

Screenshot of Two Use Cases

NOTE: If you reinstall the first reference implementation, you must also reinstall the second reference implementation. 

 

Summary and Next Steps

This application successfully implements Intel® Distribution of OpenVINO™ toolkit plugins to detect and extract the license plate information.

As a next step, try the following:

It can be extended further to provide support for feed from network stream (RTSP camera), and the algorithm can be optimized for better performance.


Learn More

To continue your learning, see the following guides and software resources:


Known Issues

Uninstall Reference Implementation

If you uninstall one of the reference implementations, you need to reinstall the other reference implementations because the Docker images will be cleared. 

 

License Plate Recognition Neural Network Issue

The neural network model used in this RI has 3 limitations:

  • It has been trained to recognize Chinese license plates.
  • The minimum plate width must be at least 94 pixels.
  • Only "blue" license plates have been tested thoroughly. Other types of license plates may underperform.

For more details, check OpenVINO™ Model Zoo.

 


Troubleshooting

Installation Failure

If the host system already has Docker images and its containers running, you will have issues during the RI installation. You must stop/force stop existing containers and images.

  • To remove all stopped containers, dangling images, and unused networks: 

    sudo docker system prune --volumes
  • To stop Docker containers: 

    sudo docker stop $(sudo docker ps -aq)
  • To remove Docker containers:

    sudo docker rm $(sudo docker ps -aq)
  • To remove all Docker images:

    sudo docker rmi -f $(sudo docker images -aq)

Docker Image Build Failure

If Docker image build on corporate network fails, follow the steps below.

  1. Get DNS server using the command: 
    nmcli dev show | grep 'IP4.DNS'
  2. Configure Docker to use the server. Paste the line below in the  /etc/docker/daemon.json file:
    { "dns": ["<dns-server-from-above-command>"]} 
  3. Restart Docker: 
    sudo systemctl daemon-reload && sudo systemctl restart docker

Installation Failure Due to Ubuntu Timezone Setting 

While building the reference implementation, if you see  /etc/timezone && apt-get install -y tzdata && ln -sf /usr/share/zoneinfo/${HOST_TIME_ZONE} /etc/localtime && dpkg-reconfigure -f noninteractive tzdata' returned a non-zero code: 1 make: *** [config] Error 1 

Run the following command in your terminal: 

sudo timedatectl set-local-rtc 0

Installation Encoding Issue 

While building the reference implementation, if you see ERROR: 'latin-1' codec can't encode character '\u2615' in position 3: ordinal not in range(256) 

Run the following command in your terminal: 

export LANG=en_US.UTF-8

Can't Connect to Docker Daemon

If you can't connect to Docker Daemon at http+docker://localhost, run the following command in your terminal: 

sudo usermod -aG docker $USER

Log out and log in to Ubuntu.  

Check before retrying to install if group Docker is available for you by running the following command in a terminal: 

groups

The output should contain docker.

Support Forum

If you're unable to resolve your issues, contact the Support Forum.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.