Intel® Edge Software Device Qualification (Intel® ESDQ) Overview

Version: 7.0   Published: 04/19/2021  

Last Updated: 01/26/2022

Overview

The Intel® Edge Software Device Qualification (ESDQ) package comes with the Command Line Interface (CLI) tool. It provides customers with the capability to run an Intel-provided test suite at the target system, with the goal of enabling partners to test their platform for a specific Intel® Edge Software package. 

Refer to the Intel® Edge Software Recommended Hardware Program Overview for more details.

For information on specific Intel® ESDQ Test Packages or to Configure & Download Intel® ESDQ Test Packages, follow the respective links below:


Time to Complete

Programming
Language

Intel® ESDQ CLI tool binary

 

 

Test Modules

Approximately 45 mins for installation and 30 mins of execution
depending on the test modules

Python* package using:  

  • Python 3.6 for Ubuntu* 18.04 

  • Python 3.8 for Ubuntu 20.04 

  • Python 3.6 for Debian* 10

Depends on the Test Modules


Target System Requirements

  • At least 16 GB RAM. 
  • At least 64 GB hard drive. 
  • A stable Internet connection. 
  • One of the below Operating Systems:
    • Ubuntu* 18.04 LTS with Kernel 5.4  
    • Ubuntu 20.04 LTS with Kernel 5.4 or 5.8 
    • Debian 10

How It Works

Intel® ESDQ package consists of Intel® ESDQ CLI binary and the respective test module Interfaces. This package is downloaded by Edge Software CLI onto the target system.

Intel® ESDQ CLI binary and its test module dependencies will be listed in the edgesoftware-configuration.xml. Edge Software CLI will use the same mechanism to download and install the Intel® ESDQ components and test suite onto the target system along with the other package components. Once downloaded, the Intel® ESDQ CLI tool can launch various test modules corresponding to the package.

The test modules interact with the Intel® ESDQ CLI through a common test module interface layer (TMI) which is part of the Intel® ESDQ CLI binary. Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which the customer can manually choose to email to the ESH support team.  

Architecture Diagram


Get Started 

Intel® ESDQ CLI tool and test modules are installed as part of the Edge Software package selected above.

NOTE: Be aware that screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release.

Download and Install Intel® ESDQ

NOTE: If you are using a Debian system, make sure to export the environment below before using the CLI:

export LC_ALL=C.UTF-8 
export LANG=C.UTF-8

Select Configure & Download on one of the pages below to download the package:

 

NOTE: The latest Intel® ESDQ Download version is 7.0. 

1. Navigate to the edgesoftware_configuration.xml included in the edgesoftware zip package and look for the Intel® ESDQ ingredient and test modules.

2. Run the command below to install the Intel® ESDQ and test modules on the target system.

./edgesoftware install 

3. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module.

NOTE: Make sure all the modules installed successfully before you start using the Intel® ESDQ. If any module failed to install, run the command ./edgesoftware update <Failed Module ID(s)>

4. Once the installation is completed, reboot the system.

5. The Intel® ESDQ CLI and the test modules are already available in the target system. To go to the directory for the Intel® ESDQ, edit the change directory command below before running it:

cd <Installation Directory>/edge_software_device_qualification/Edge_Software_Device_Qualification_7.0/esdq

Run the Application

NOTE: If you are using a Debian system, make sure to export the environment below before using the CLI:

export LC_ALL=C.UTF-8 
export LANG=C.UTF-8

1. Change directory to the location of Intel® ESDQ folder using the command:

cd <esdq Location> 

2. Check the Intel® ESDQ version using the command: 

./esdq --version OR ./esdq -v

Ensure the release version is shown as in this example:

3. Run Intel® ESDQ Help:

./esdq --help

Screenshot of system info

4. Run Intel® ESDQ System Info to show a snapshot of the system OS version, kernel, and hardware info such as device manufacturer, processor architecture and type, memory size and hard disk drive configurations. This command also gives information about hardware accelerators, if any. 

./esdq systeminfo

5. Run Intel® ESDQ List to get the list of supported test modules: 

./esdq list

Example output:

Screenshot of test modules

6. Run Intel® ESDQ test execution. This command runs the test modules, which are listed in the package. 

./esdq run

NOTE: If the tool asks for super user permission, please provide the sudo password when prompted.

7. Run Intel® ESDQ Test execution and generate the report:

./esdq run --help

Screenshot of generate report

8. The commands below will start executing all the test modules available in the list and generate the report. 

./esdq run –r

 Or use the following command:  

./esdq run --report

 Or you can also selectively execute the tests from your interested Test Modules by passing the Module ID to run the command as shown below. You can get the Module ID from ./esdq list command. 

./esdq run [MODULE_ID]

Example: 

./esdq run 60f08135acbb0a002bcaee70

9. Once the tests are completed, the intermediate artifacts are available in the esdqReport folder.

10. If you did not use the –r option for the run command, you can generate a report separately using the Intel® ESDQ Report Generation command:

./esdq genreport  

This command generates the final report in HTML format and prepares the zip package with all the required artifacts. The HTML report contains a Summary section, System Info section, and a neatly formatted test module report section. 

Zip Package Example: esdqReport_2020-07-28_00:29:19.zip

The final report should look like the following example:

Screenshot of test summary

Screenshot of test summary modules info

Screenshot of test suites

Screenshot of vats tests

 

NOTE: The screenshot above is an illustration of how Intel® ESDQ displays the pass and fail status and is not indicative of the release state of test modules.  

11. Navigate to the artifacts location by clicking on the respective test module name as shown below.

Screenshot of navigation to test suite artifacts

 


Release Notes 

Current Version: 7.0   

New in this Release   

  • Incorporated customer feedback from previous release.     
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI report shows Intel® ESDQ CLI Version

Version: 6.0   

New in this Release   

Incorporated customer feedback from previous 5.0 release.     

  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI now supports parameters that can be passed to Test Modules. 
  • Intel® ESDQ CLI now supports up to two reports per Test Module.

Version: 5.0   

New in this Release   

Incorporated customer feedback from previous 4.0 release.     

  • Incorporated customer feedback from previous 4.0 release.     
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI shows more details for version info. 
  • Intel® ESDQ CLI collects all the ESH installation logs as part of esdq final artifact zip file.

Version: 4.0  

New in this Release  

  • Incorporated customer feedback from previous 3.0 release.    
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.   
  • Intel® ESDQ CLI supports Debian 10 in addition to Ubuntu 20.04 & 18.04. 
  • Intel® ESDQ Report now captures Module info to show the installation status.   
  • Intel® ESDQ CLI adds options to execute only interested Test Modules.  
  • Intel® ESDQ CLI reports now support the color code for each test case.  
  • Intel® ESDQ CLI shows more details for version info. 

Version: 3.0 

New in this Release 

  • Incorporated customer feedback from previous 2.0 release.   
  • Intel® ESDQ 3.0 integrates Intel® VATS and Edge Insights for Vision Test Module with OpenVINO™ toolkit 2021.3 support. 
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.  
  • Intel® ESDQ CLI supports Ubuntu 20.04 and 18.04.  
  • Intel® ESDQ CLI System Info command displays GPU information.  
  • Intel® ESDQ CLI adds a new list command to list the supported test modules.  
  • Intel® ESDQ CLI shows more details for version info.

Known issues 

  • Upgrade from EDTDT 2.0 to Intel® ESDQ 3.0 is not supported and a fresh install of Intel® ESDQ 3.0 is required. 

Version: 2.0 

New in this Release 

  • Incorporated feedback from customer from previous 1.0 release.  
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used. 
  • CLI supports Ubuntu 20.04 in addition to Ubuntu 18.04. 

Known issues 

  • Report generation in PDF has been deprecated in version 2.0 due to known issues. 

Version: 1.0 

New in this Release 

  • Initial Features for Recommended Configuration. 
  • Installs all the prerequisites and dependencies for the Edge Device Test and Diagnostic Tool. 
  • Supports below use cases: 
    • Run – launches the Vision test module and Intel® VATS test module. If there are no Vision accelerator cards present in the system, the HDDL test cases will fail. 
    • System info – collects the system information where the test is run. 
    • genreport – Generates a summary test report in html and pdf for all the test modules that were executed.  

Known Issues 

  • The pdf type of report is not showing the color format for Status.   

Summary and Next Steps

With the Intel® ESDQ CLI tool, you were able to run an Intel provided test suite at the target system, with the goal of testing your platform for a specific Edge Software package. Intel® ESDQ generated a complete test report in HTML format, along with detailed logs packaged as one zip file. 

As a next step, you can submit the final result package (Example: esdqReport_2021-05-05_07:39:19.zip) by email to the eshsupport@intel.com.


Troubleshooting

Incompatible Python Version  

Use the command below to display the esdq cli version, which shows the operating system the binary supports. 

./esdq --version OR ./esdq -v

Example:

Screenshot of release version

esdq genreport Command Error

When a user invokes the command esdq genreport  the following error is reported: 

      Exiting ESDQ Execution !! Make sure to run the tests using 'esdq run' before generating the report. 

Debian 10 Python Environment Issue 

You may encounter the RuntimeError below when executing esdq or edgesoftware cli binary.

Screenshot of environment issue

  • Solution:Execute the exports belowbefore CLI usage: 

    export LC_ALL=C.UTF-8 
    export LANG=C.UTF-8 

Intel® ESDQ Package Installation in Debian 10 Platform 

Follow the instructions under the section Get Started on the Intel® Edge Software Device Qualification (Intel® ESDQ) - Real Time Performance Measurement (RTPM) page.

Support Forum

If you're unable to resolve your issues, contact the Support Forum

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.