Intel® Edge Software Device Qualification (Intel® ESDQ) Overview

ID 658268
Updated 10/26/2022
Version 10.0
Public

author-image

By

Overview

The Intel® Edge Software Device Qualification (ESDQ) package comes with the Command Line Interface (CLI) tool. It provides customers with the capability to run an Intel-provided test suite at the target system, with the goal of enabling customers to test their platform for a specific Intel® Edge Software package. 

Refer to the Intel® Edge Software Recommended Hardware Program Overview for more details.

For information on specific Intel® ESDQ Test Packages or to Configure & Download Intel® ESDQ Test Packages, follow the respective links below:

 

Time to Complete: Approximately 45 mins for installation and 30 mins of execution depending on the test modules

Programming Language: Python* package using:  

  • Python 3.6 for Ubuntu* 18.04 
  • Python 3.8 for Ubuntu 20.04 
  • Python 3.8 for Debian* 11

Test Modules: Depends on the Test Modules

Target System Requirements

  • At least 16 GB RAM. 
  • At least 64 GB hard drive. 
  • A stable Internet connection. 
  • One of the below Operating Systems:
    • Ubuntu* 18.04 LTS with Kernel 5.4  
    • Ubuntu 20.04 LTS with Kernel 5.4 or 5.8 
    • Debian 11

How It Works

Intel® ESDQ package consists of Intel® ESDQ CLI binary and the respective test module Interfaces. This package is downloaded by Edge Software CLI onto the target system.

Intel® ESDQ CLI binary and its test module dependencies will be listed in the edgesoftware-configuration.xml. Edge Software CLI will use the same mechanism to download and install the Intel® ESDQ components and test suite onto the target system along with the other package components. Once downloaded, the Intel® ESDQ CLI tool can launch various test modules corresponding to the package.

The test modules interact with the Intel® ESDQ CLI through a common test module interface layer (TMI) which is part of the Intel® ESDQ CLI binary. Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which the customer can manually choose to email to the ESH support team.  

Block diagram that starts with a box for Intel ESDQ CLI with arrows to a box for the Test Modules. An arrow points from there to Intel ESDQ Report Generation with an image of a computer monitor.

Get Started 

Intel® ESDQ CLI tool and test modules are installed as part of the Edge Software package selected above.

NOTE: Be aware that screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release.

Download and Install Intel® ESDQ

NOTE: If you are using a Debian system, make sure to export the environment below before using the CLI:
export LC_ALL=C.UTF-8 
export LANG=C.UTF-8

Select Configure & Download on one of the pages below to download the package:

 

NOTE: The latest Intel® ESDQ Download version is 10.0. 

 

  1. Navigate to the edgesoftware_configuration.xml included in the edgesoftware zip package and look for the Intel® ESDQ ingredient and test modules.
  2. Run the command below to install the Intel® ESDQ and test modules on the target system. 
    ./edgesoftware install 
  3. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module.
    NOTE: Make sure all the modules installed successfully before you start using the Intel® ESDQ. If any module failed to install, run the command: ./edgesoftware update <Failed Module ID(s)>
     
  4. Once the installation is completed, reboot the system.
  5. The Intel® ESDQ CLI and the test modules are already available in the target system. To go to the directory for the Intel® ESDQ, edit the change directory command below before running it: 
    cd <Installation Directory>/edge_software_device_qualification/Edge_Software_Device_Qualification_<version>/esdq
    Where <version> indicates the software version downloaded.
     

Run the Application

NOTES: 
If you are using a Debian system, make sure to export the environment below before using the CLI:
export LC_ALL=C.UTF-8 
export LANG=C.UTF-8

Install usbutils, pciutils, unzip, and python3-psutil as prerequisites for ESDQ functionality with the commands: 
apt install usbutils pciutils
unzip python3-psutil

 

  1. Change directory to the location of Intel® ESDQ folder using the command: 
    cd <esdq Location> 
  2. Check the Intel® ESDQ version using the command:  
    ./esdq --version OR ./esdq -v

    Ensure the release version matches the version you downloaded. The example below is from an earlier release.  
    esdq, version 5.0, Build date: 2021-09-21, Target OS: Ubuntu 18.04; Debian
  3. Run Intel® ESDQ Help: 
    ./esdq --help

    Screenshot of a terminal with the help command available with a list of commands beneath it.
     
  4. Run Intel® ESDQ System Info to show a snapshot of the system OS version, kernel, and hardware info such as device manufacturer, processor architecture and type, memory size and hard disk drive configurations. This command also gives information about hardware accelerators, if any.  
    ./esdq systeminfo
  5. Run Intel® ESDQ List to get the list of supported test modules:  
    ./esdq list

    Example output:
    Screenshot of test modules with their version and ID.
     
  6. Run Intel® ESDQ test execution. This command runs the test modules, which are listed in the package.  
    ./esdq run

    NOTE: If the tool asks for super user permission, please provide the sudo password when prompted.
     
  7. Run Intel® ESDQ Test execution and generate the report: 
    ./esdq run --help

    Screenshot of terminal with help command as an option.
     
  8. The commands below will start executing all the test modules available in the list and generate the report.  
    ./esdq run –r

    Or use the following command:   
    ./esdq run --report

    Or you can also selectively execute the tests from your interested Test Modules by passing the Module ID or Module Name to run the command as shown below.
    You can get the Module ID or Module Name from the ./esdq list command.  
    ./esdq run [MODULE_ID or MODULE_NAME]

    Examples:  
    ./esdq run 60f08135acbb0a002bcaee70
./esdq run Sample1_Test_Module
     

    Or you can also execute the tests from your interested Test Module by passing the parameters to the test module if the test module supports it. 

    ./esdq run [MODULE_ID or MODULE_NAME] -p [PARAMETERS]


    Examples: 

    ./esdq run 60f08135acbb0a002bcaee70 –p "-c"
./esdq run Sample1_Test_Module –p "-c"
./esdq run -r –p "sample"


    NOTE: If the package has more than 1 test module, then you also need to pass UUID or Test Module Name. At any point, only one test module can be executed by passing parameters.

  9. Once the tests are completed, the intermediate artifacts are available in the esdqReport folder.
  10. If you did not use the –r option for the run command, you can generate a report separately using the Intel® ESDQ Report Generation command: 
    ./esdq genreport  

    This command generates the final report in HTML format and prepares the zip package with all the required artifacts. The HTML report contains a Summary section, System Info section, and a neatly formatted test module report section. 
    Zip Package Example: esdqReport_2020-07-28_00:29:19.zip
    The final report should look like the following example:
    Screenshot of the Test Summary that contains a summary and SystemInfo.

    Screenshot of the Test Summary that contains the ModulesInfo.

    Screenshot of the Test Summary that contains the Test Suites with all status listed as "Pass."

    Screenshot of the Test Summary that contains Sample2_Test_Module with all status listed as "Pass."

    NOTE: The screenshot above is an illustration of how Intel® ESDQ displays the pass and fail status and is not indicative of the release state of test modules.  
     
  11. Navigate to the artifacts location by clicking on the respective test module name as shown below.
    Screenshot of the Test Summary that contains Sample1_Test_Module with all status listed as "Pass."

 

 

Release Notes 

Current Version: 10.0

New in this Release  

  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.
  • Intel® ESDQ CLI report captures installed modules version information under Module Info table.
  • Intel® ESDQ CLI is enhanced to support the individual test module execution by using the test module name. In earlier versions, ESDQ CLI used to support only through UUID.

Version: 9.0

New in this Release   

  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.
  • Intel® ESDQ CLI report now supports up to 4 tables per test module.
  • Intel® ESDQ CLI is enhanced to support parameters for both standalone and UUID based execution.
  • Intel® ESDQ CLI is enhanced to support hyperlinks for logs navigation as part of test reporting.

Version: 8.0

New in this Release   

  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.
  • Intel® ESDQ CLI report summary section is updated to capture recipe name, recipe version. 
  • Intel® ESDQ CLI report summary section is updated to show the test categories (Test Module) in tabular form.
  • Intel® ESDQ CLI is enhanced to support the display of up to two images per test module as part of its reporting.
  • Intel® ESDQ CLI version is made consistent across all test modules by removing minor versions. 

Version: 7.0   

New in this Release   

  • Incorporated customer feedback from previous release.     
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI report shows Intel® ESDQ CLI Version

Version: 6.0   

New in this Release   

Incorporated customer feedback from previous 5.0 release.     

  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI now supports parameters that can be passed to Test Modules. 
  • Intel® ESDQ CLI now supports up to two reports per Test Module.

Version: 5.0   

New in this Release   

Incorporated customer feedback from previous 4.0 release.     

  • Incorporated customer feedback from previous 4.0 release.     
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.    
  • Intel® ESDQ CLI shows more details for version info. 
  • Intel® ESDQ CLI collects all the ESH installation logs as part of esdq final artifact zip file.

Version: 4.0  

New in this Release  

  • Incorporated customer feedback from previous 3.0 release.    
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.   
  • Intel® ESDQ CLI supports Debian 10 in addition to Ubuntu 20.04 & 18.04. 
  • Intel® ESDQ Report now captures Module info to show the installation status.   
  • Intel® ESDQ CLI adds options to execute only interested Test Modules.  
  • Intel® ESDQ CLI reports now support the color code for each test case.  
  • Intel® ESDQ CLI shows more details for version info. 

Version: 3.0 

New in this Release 

  • Incorporated customer feedback from previous 2.0 release.   
  • Intel® ESDQ 3.0 integrates Intel® VATS and Edge Insights for Vision Test Module with OpenVINO™ toolkit 2021.3 support. 
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used.  
  • Intel® ESDQ CLI supports Ubuntu 20.04 and 18.04.  
  • Intel® ESDQ CLI System Info command displays GPU information.  
  • Intel® ESDQ CLI adds a new list command to list the supported test modules.  
  • Intel® ESDQ CLI shows more details for version info.

Known Issues 

  • Upgrade from EDTDT 2.0 to Intel® ESDQ 3.0 is not supported and a fresh install of Intel® ESDQ 3.0 is required. 

Version: 2.0 

New in this Release 

  • Incorporated feedback from customer from previous 1.0 release.  
  • Intel® ESDQ README.txt is updated to capture the version information of the Python modules being used. 
  • CLI supports Ubuntu 20.04 in addition to Ubuntu 18.04. 

Known Issues 

  • Report generation in PDF has been deprecated in version 2.0 due to known issues. 

Version: 1.0 

New in this Release 

  • Initial Features for Recommended Configuration. 
  • Installs all the prerequisites and dependencies for the Edge Device Test and Diagnostic Tool. 
  • Supports below use cases: 
    • Run – launches the Vision test module and Intel® VATS test module. If there are no Vision accelerator cards present in the system, the HDDL test cases will fail. 
    • System info – collects the system information where the test is run. 
    • genreport – Generates a summary test report in html and pdf for all the test modules that were executed.  

Known Issues 

  • The pdf type of report is not showing the color format for Status.   

Summary and Next Steps

With the Intel® ESDQ CLI tool, you were able to run an Intel provided test suite at the target system, with the goal of testing your platform for a specific Edge Software package. Intel® ESDQ generated a complete test report in HTML format, along with detailed logs packaged as one zip file. 

As a next step, you can submit the final result package (Example: esdqReport_2021-05-05_07:39:19.zip) by email to the eshsupport@intel.com.

Troubleshooting

Incompatible Python Version  

Use the command below to display the esdq cli version, which shows the operating system the binary supports. 

./esdq --version OR ./esdq -v

Example:

Screenshot of release version, build date and target OS.

esdq genreport Command Error

When a user invokes the command esdq genreport  the following error is reported: 

      Exiting ESDQ Execution !! Make sure to run the tests using 'esdq run' before generating the report. 

Debian 10 Python Environment Issue 

You may encounter the RuntimeError below when executing esdq or edgesoftware CLI binary.

Screenshot of the RuntimeError

  • Solution:Execute the exports below before CLI usage: 

    export LC_ALL=C.UTF-8 
    export LANG=C.UTF-8 

Intel® ESDQ Package Installation in Debian 10 Platform 

Follow the instructions under the section Get Started on the Intel® Edge Software Device Qualification (Intel® ESDQ) - Real Time Performance Measurement (RTPM) page.

Support Forum

If you're unable to resolve your issues, contact the Support Forum. 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.