Intel® Edge Software Device Qualification (Intel® ESDQ) for AI Box for Video Analytics

Version: 2.0   Published: 09/29/2021  

Last Updated: 04/01/2022

Overview

Intel® Edge Software Device Qualification (Intel® ESDQ) for Edge AI Box for Video Analytics provides customers with the capability to run an Intel provided test suite at the target system, with the goal of enabling partners to self-qualify on their platform.   

The information below is specific to the Intel® ESDQ for Edge AI Box for Video Analytics package. For documentation on the Intel® ESDQ CLI binary, refer to Intel® Edge Software Device Qualification (Intel® ESDQ) Overview.

Select Configure & Download to download Intel® ESDQ for Edge AI Box for Video Analytics and refer to Get Started for installation steps.  

Configure & Download


Target System Requirements 

  • 11th or 12th Generation Intel® Core™ processors and Celeron® processors
  • Ubuntu* 20.04.4 with Kernel Version 5.16 or newer 
  • At least 128GB disk space 
  • At least 8GB (4GB x2) memory with 8GB swap space 
  • Direct Internet Access
  • A stable internet connection

 


How It Works

The test module for Edge AI Box for Video Analytics interacts with the Intel® ESDQ CLI through a common test module interface (TMI) layer, which is part of the Intel® ESDQ binary.

Selected components from customized download options will be validated. Execute the automated test suite. 

Test results will be stored in the output folder. Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which you can manually choose to email to the ESH support team.

AI Box Test Module 

The AI Box test module is a validation framework for Edge AI Box for Video Analytics. It validates the installation status as part of health check. The test module includes test cases for Intel® Media SDK and Intel® Distribution of OpenVINO™ toolkit 2021.4 Runtime and YOLO v3 benchmark performance test as shown below: 

  • mediasdk_sample_multi_transcode with h264 and mpeg2
  • mediasdk_set_commands with h264 mixed model
  • mediasdk_set_commands with h265 mixed model
  • mediasdk_sample_encode with h265
  • mediasdk_sample_vpp with sw
  • hello_reshape_ssd with CPU
  • style_transfer_sample_python with CPU
  • object_detection_demo_ssd_async CPU
  • object_detection_demo_python with CPU
  • hello_reshape_ssd with GPU
  • style_transfer_sample_python with GPU
  • object_detection_demo_ssd_async GPU
  • run_tc_object_detection_demo_ssd__python with GPU
  • ai_box_benchmark
  • ai_box_svet
  • tensorflow version 
  • openvino-tensorflow version 
  • azure PnP bridge repo sample path 
  • azure cli 
  • azure explorer 
  • aws Greengrass 
  • aws cli

Health Check

The test suite runs a health check to make sure all the components that are part of the vision package listed below are properly installed and configured correctly. 

  • Reference Implementation - Multi-Camera Detection for Social Distancing
  • Intel® Distribution of OpenVINO™ toolkit 2021.4 Runtime 
  • Intel® Edge Software Device Qualification

Media SDK Test 

Intel® Media SDK enables hardware acceleration for fast video transcoding, image processing and media workflows. For each of the Intel® Media SDK components, the test suite runs a series of sample applications such as decode, transcode, encode with respect to CPU and GPU. The test suite validates the output of the sample applications to make sure that all the components work as expected.

OpenVINO™ Test

For each of the Intel® OpenVINO™ toolkit components, the test suite runs a series of sample applications such as style transfer, reshape and object detection on CPU and GPU. The test suite validates the output of the sample applications to make sure that all the components work as expected.

Benchmark Performance Test

The benchmark tests demonstrate high performance gains on several public neural networks on multiple Intel® CPUs and GPUs covering a broad performance range. Use this data to help decide which hardware is best for applications and solutions, or to plan AI workload on the Intel computing already included in the solutions. The Python* based benchmark test performs inference using convolutional networks. More details about the benchmark tool can be found here.

The OpenVINO™ Benchmark application is used to check whether INT8 network models of ResNet-50-tf and YOLO-v3-tf can be executed on Intel® GPUs with qualified performance. The YOLO-v3-tf is a popular network used in object detection scenarios. The ResNet-50-tf is a popular backbone both in detection networks and classification networks. The expected throughput performance of processors can be found at Intel® Distribution of OpenVINO™ toolkit Benchmark Results.  

You can see the benchmark application test results directly in a CSV file (AIBox_csvReport.out) under the output folder. The complete execution log of benchmark app tests can be found in the AIBox_BmTestOutput.out file.  The actual inference throughput results can be read in the log file. The CSV file output information will appear as shown below.

Screenshot of Benchmark results in CSV file

Figure 1. Benchmark Results in CSV File

 

Smart Video and AI Workload Reference Implementation Test 

The test sources the OpenVINO™ and Smart Video and AI Workload environment setup and executes the video_e2e_sample with the par_file entry. To run the video_e2e_sample, the script prompts for the super user password. This test performs the face detection scenario with the 16 channels using the 1080p.h264 video. The output is then collected in the log file.

Get Started 

Intel® ESDQ 6.0 CLI tool is installed as part of the Edge AI Box for Video Analytics package. 

NOTE: Be aware that screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release.

Download and Install Intel® ESDQ for Edge AI Box for Video Analytics

1. Select Configure & Download to download Intel® ESDQ Edge AI Box for Video Analytics and then follow the steps below to install it. Click on Customize download to select the configuration.

Configure & Download

Configure and download dialog

Figure 2. Configure & Download Page

 

2. Choose the required configuration based on your test system environment. We recommend you select Select Version or Tag as 2.0, Target System as Ubuntu 20.04 LTS, Environment as Development, and Distribution as Customize Download on the displayed options screen.

3. Click the Customize Download button to download the Intel® Edge Software Device Qualification (Intel® ESDQ) for Edge AI Box for Video Analytics.

Customize Download dialog

Figure 3. Select Download

 

4. Click the Edit button to reconfigure the options and download another version of Intel® ESDQ for AI Box for Video Analytics in subsequent releases.

Edit Download dialog

Figure 4. Edit Download

 

5. Click the Next button until the Download button appears (screen 7 of 7). 

Screenshot of Customize Download 1 of 6

Figure 5. Customize Download - Screen 1

 

6. You will notice one of the components in the list is Intel® Edge Software Device Qualification. Click the Download button.

Screenshot of Customize Download 6 of 6

Figure 6. Customize Download - Screen 6

 

7. Select Accept License Agreement on the pop-up window, then follow the instructions below.

8. Copy and Save the Product Key

Product Key

Figure 7. Product Key

 

9. Transfer the downloaded package to the target Ubuntu* system.

10. View the edgesoftware_configuration.xml included in the edge software zip package and look for the Intel® ESDQ ingredient and test modules for Edge AI Box for Video Analytics. 

<project path="installation/src" id="6226715231577d002df8b7d9" version="2.0" label="esdq/Aibox_Test_Module" /> 
<project path="installation/src" id="6226827031577d002df8b7f4" version="7.0.1" label="esdq" esb_install="true" />  

 

11. Prepare the target system for Edge AI Box for Video Analytics installation: 

  • Make sure the target system is Ubuntu 20.04, Linux Kernel 5.8 or above.
  • Make sure apt source list is updated to latest and sudo, lsusb, unzip are already installed. 
    apt update 
    apt install sudo usbutils unzip 
  • Make sure the Python 3 environment is set up correctly. 
  • Make sure pip3 setup tools, psutil, are available in the target system.  
    pip3 install –-upgrade pip 
    pip3 install setuptools psutil

 

12. Provide executable permission to edgesoftware

chmod +x ./edgesoftware

 

13. Run the command below to install Intel® ESDQ 6.0 and Edge AI Box for Video Analytics. 

./edgesoftware install

 

14. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download (or from the saved Product Key in the step above). 

15. When the installation is complete, you see the message Installation of package complete and the installation status for each module. 

Screenshot of Installation Status

Figure 8. Installation Status

 

16. Find the module ID for the Intel® ESDQ by running the command: 

./edgesoftware list --default

  

Example output:  

Screenshot of Edgesoftware List

Figure 9. Edgesoftware List

17. After module installation is completed successfully, reboot the system:

reboot

  


Run the Application

For the complete Intel® ESDQ CLI, refer to Intel® ESDQ CLI Overview.  The test modules are already available in the target system.

1. Change directory with the command: 

cd $HOME/edge_ai_box_for_video_analytics/Edge_AI_Box_for_Video_Analytics_2.0/esdq

2. Run Intel® ESDQ test and generate report: 

./esdq run –r 

3. When executing the Smart Video and AI workload Reference Implementation, enter the password of super user. 

Password:

The test report should look like the following examples:

Note: The results shown below are for illustration only.

Test Report Summary

 

 

Test Report Modules Info

 

Test Report Test Suites

 

 


Release Notes 

Current Version: 2.0 

New in This Release   

  • Integrated OpenVINO™ 2021.4.2. 
  • Included the functionality test of Azure IoT PnP Bridge, OpenVINO™ TensorFlow Bridge and Amazon Web Services Greengrass.

Version: 1.2

New in This Release  

  • Latency and Throughput information included in HTML report. 
  • Included the functionality test of Smart Video and AI Workload Reference Implementation.

Version 1.0

New in This Release  

  • Initial features for recommended configuration. 

Known Issues 

  • If yolo-v3-tf.xml is not downloaded, then Latency and Throughput metrics will be null in the Intel® ESDQ HTML report page.

Support Forum

If you're unable to resolve your issues, contact the Support Forum

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.