Intel® Edge Software Device Qualification (Intel® ESDQ) for Edge AI Box for Video Analytics provides customers with the capability to run an Intel provided test suite at the target system, with the goal of enabling partners to self-qualify on their platform.
The information below is specific to the Intel® ESDQ for Edge AI Box for Video Analytics package. For documentation on the Intel® ESDQ CLI binary, refer to Intel® Edge Software Device Qualification (Intel® ESDQ) Overview.
Target System Requirements
- 11th or 12th Generation Intel® Core™ processors and Celeron® processors
- Ubuntu* 20.04.4 with Kernel Version 5.16 or newer
- At least 128GB disk space
- At least 8GB (4GB x2) memory with 8GB swap space
- Direct Internet Access
- A stable internet connection
How It Works
The test module for Edge AI Box for Video Analytics interacts with the Intel® ESDQ CLI through a common test module interface (TMI) layer, which is part of the Intel® ESDQ binary.
Selected components from customized download options will be validated. Execute the automated test suite.
Test results will be stored in the output folder. Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which you can manually choose to email to the ESH support team.
AI Box Test Module
The AI Box test module is a validation framework for Edge AI Box for Video Analytics. It validates the installation status as part of health check. The test module includes test cases for Intel® Media SDK and Intel® Distribution of OpenVINO™ toolkit 2021.4 Runtime and YOLO v3 benchmark performance test as shown below:
- mediasdk_sample_multi_transcode with h264 and mpeg2
- mediasdk_set_commands with h264 mixed model
- mediasdk_set_commands with h265 mixed model
- mediasdk_sample_encode with h265
- mediasdk_sample_vpp with sw
- hello_reshape_ssd with CPU
- style_transfer_sample_python with CPU
- object_detection_demo_ssd_async CPU
- object_detection_demo_python with CPU
- hello_reshape_ssd with GPU
- style_transfer_sample_python with GPU
- object_detection_demo_ssd_async GPU
- run_tc_object_detection_demo_ssd__python with GPU
- tensorflow version
- openvino-tensorflow version
- azure PnP bridge repo sample path
- azure cli
- azure explorer
- aws Greengrass
- aws cli
The test suite runs a health check to make sure all the components that are part of the vision package listed below are properly installed and configured correctly.
- Reference Implementation - Multi-Camera Detection for Social Distancing
- Intel® Distribution of OpenVINO™ toolkit 2021.4 Runtime
- Intel® Edge Software Device Qualification
Media SDK Test
Intel® Media SDK enables hardware acceleration for fast video transcoding, image processing and media workflows. For each of the Intel® Media SDK components, the test suite runs a series of sample applications such as decode, transcode, encode with respect to CPU and GPU. The test suite validates the output of the sample applications to make sure that all the components work as expected.
For each of the Intel® OpenVINO™ toolkit components, the test suite runs a series of sample applications such as style transfer, reshape and object detection on CPU and GPU. The test suite validates the output of the sample applications to make sure that all the components work as expected.
Benchmark Performance Test
The benchmark tests demonstrate high performance gains on several public neural networks on multiple Intel® CPUs and GPUs covering a broad performance range. Use this data to help decide which hardware is best for applications and solutions, or to plan AI workload on the Intel computing already included in the solutions. The Python* based benchmark test performs inference using convolutional networks. More details about the benchmark tool can be found here.
The OpenVINO™ Benchmark application is used to check whether INT8 network models of ResNet-50-tf and YOLO-v3-tf can be executed on Intel® GPUs with qualified performance. The YOLO-v3-tf is a popular network used in object detection scenarios. The ResNet-50-tf is a popular backbone both in detection networks and classification networks. The expected throughput performance of processors can be found at Intel® Distribution of OpenVINO™ toolkit Benchmark Results.
You can see the benchmark application test results directly in a CSV file (AIBox_csvReport.out) under the output folder. The complete execution log of benchmark app tests can be found in the AIBox_BmTestOutput.out file. The actual inference throughput results can be read in the log file. The CSV file output information will appear as shown below.
Figure 1. Benchmark Results in CSV File
Smart Video and AI Workload Reference Implementation Test
The test sources the OpenVINO™ and Smart Video and AI Workload environment setup and executes the video_e2e_sample with the par_file entry. To run the video_e2e_sample, the script prompts for the super user password. This test performs the face detection scenario with the 16 channels using the 1080p.h264 video. The output is then collected in the log file.
Intel® ESDQ 6.0 CLI tool is installed as part of the Edge AI Box for Video Analytics package.
NOTE: Be aware that screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release.
Download and Install Intel® ESDQ for Edge AI Box for Video Analytics
1. Select Configure & Download to download Intel® ESDQ Edge AI Box for Video Analytics and then follow the steps below to install it. Click on Customize download to select the configuration.
Figure 2. Configure & Download Page
2. Choose the required configuration based on your test system environment. We recommend you select Select Version or Tag as 2.0, Target System as Ubuntu 20.04 LTS, Environment as Development, and Distribution as Customize Download on the displayed options screen.
3. Click the Customize Download button to download the Intel® Edge Software Device Qualification (Intel® ESDQ) for Edge AI Box for Video Analytics.
Figure 3. Select Download
4. Click the Edit button to reconfigure the options and download another version of Intel® ESDQ for AI Box for Video Analytics in subsequent releases.
Figure 4. Edit Download
5. Click the Next button until the Download button appears (screen 7 of 7).
Figure 5. Customize Download - Screen 1
6. You will notice one of the components in the list is Intel® Edge Software Device Qualification. Click the Download button.
Figure 6. Customize Download - Screen 6
7. Select Accept License Agreement on the pop-up window, then follow the instructions below.
8. Copy and Save the Product Key.
Figure 7. Product Key
9. Transfer the downloaded package to the target Ubuntu* system.
10. View the edgesoftware_configuration.xml included in the edge software zip package and look for the Intel® ESDQ ingredient and test modules for Edge AI Box for Video Analytics.
<project path="installation/src" id="6226715231577d002df8b7d9" version="2.0" label="esdq/Aibox_Test_Module" /> <project path="installation/src" id="6226827031577d002df8b7f4" version="7.0.1" label="esdq" esb_install="true" />
11. Prepare the target system for Edge AI Box for Video Analytics installation:
- Make sure the target system is Ubuntu 20.04, Linux Kernel 5.8 or above.
- Make sure apt source list is updated to latest and sudo, lsusb, unzip are already installed.
apt update apt install sudo usbutils unzip
- Make sure the Python 3 environment is set up correctly.
- Make sure pip3 setup tools, psutil, are available in the target system.
pip3 install –-upgrade pip pip3 install setuptools psutil
12. Provide executable permission to edgesoftware.
chmod +x ./edgesoftware
13. Run the command below to install Intel® ESDQ 6.0 and Edge AI Box for Video Analytics.
14. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download (or from the saved Product Key in the step above).
15. When the installation is complete, you see the message Installation of package complete and the installation status for each module.
Figure 8. Installation Status
16. Find the module ID for the Intel® ESDQ by running the command:
./edgesoftware list --default
Figure 9. Edgesoftware List
17. After module installation is completed successfully, reboot the system:
Run the Application
For the complete Intel® ESDQ CLI, refer to Intel® ESDQ CLI Overview. The test modules are already available in the target system.
1. Change directory with the command:
2. Run Intel® ESDQ test and generate report:
./esdq run –r
3. When executing the Smart Video and AI workload Reference Implementation, enter the password of super user.
The test report should look like the following examples:
Note: The results shown below are for illustration only.
Current Version: 2.0
New in This Release
- Integrated OpenVINO™ 2021.4.2.
- Included the functionality test of Azure IoT PnP Bridge, OpenVINO™ TensorFlow Bridge and Amazon Web Services Greengrass.
New in This Release
- Latency and Throughput information included in HTML report.
- Included the functionality test of Smart Video and AI Workload Reference Implementation.
New in This Release
- Initial features for recommended configuration.
- If yolo-v3-tf.xml is not downloaded, then Latency and Throughput metrics will be null in the Intel® ESDQ HTML report page.
If you're unable to resolve your issues, contact the Support Forum.
Product and Performance Information
Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.