Intel® Edge Software Device Qualification (Intel® ESDQ) for Intel® Edge Controls for Industrial (Intel® ECI)

Version: 2022.3   Published: 09/29/2021  

Last Updated: 09/19/2022

Overview

Intel® Edge Software Device Qualification (Intel® ESDQ) for Intel® Edge Controls for Industrial (Intel® ECI) package provides customers with the capability to run Intel provided ECI Real Time Performance Measurement (RTPM) and Time-Sensitive Networking (TSN) test suites at the target system, with the goal of enabling partners to measure their platform performances for Intel® ECI package.   

The information below is specific to the Intel® ESDQ for ECI package test modules. For documentation on the Intel® ESDQ CLI binary, refer to Intel® Edge Software Device Qualification (Intel® ESDQ) Overview.

Select Configure & Download to download Intel® ESDQ for Edge Controls for Industrial (ECI) and refer to Get Started for installation steps.  

Configure & Download


Target System Requirements 

 


How It Works

Real Time Performance Measurement (RTPM) Test Modules

Real Time Performance Measurement (RTPM) 

Real-time computing is often critical to industrial usage scenarios. Intel real-time solutions are focused on hard real-time use cases where Capability fails if cannot be executed within the allotted time span. Examples of the usage scenarios include robotics, automotive, etc.   

On a real-time computing system, there are several factors that could impact the latency of reaction to the trigger event. These factors include hardware design, BIOS configuration, OS kernel configuration, system settings, etc. RTPM is designed to check the key settings of the system and help you to identify hotspots of the system for real-time performance and give a recommendation based on Best-Known Configuration (BKC). In addition, it provides a way to measure the system scheduling latency with some open source tools.

The RTPM interacts with the Intel® ESDQ CLI through a common test module interface layer (TMI) which is part of the Intel® ESDQ binary.

Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which you can manually choose to email to the ESH support team.

RTPM contains the following test modules:

Real-Time Readiness Check 

This module leverages one of the tools in Intel® Time Coordinated Computing Tools to check the many attributes that may affect real-time performance.

 

Screenshot of real-time performance checker that includes Software SRAM dependencies checker and Real-time BIOS capabilities checkers. It shows checker name, description, and notes for each.

 

  • Verifies whether the system has a supported processor, BIOS, and OS. 
  • Checks for features that may affect real-time performance, such as Intel® Turbo Boost Technology, Enhanced Intel SpeedStep® Technology, and processor power-saving states. 
  • Reports CPU and GPU frequencies. 
  • Operates at the OS level.

Boot Command Line Check 

This module checks the Real-Time OS boot command line parameters and gives recommended settings per best known configurations.

Screenshot of Real-Time OS boot command line parameters and BKCs.

Kernel Configuration Check 

This module checks the Real-Time OS kernel configuration and gives recommendation settings per best known configurations. 

Screenshot of Real-Time OS kernel configuration check.

Real-Time Performance Test 

This module contains the below benchmarks to evaluate the performance of the target system.  

Latency
  • Latency tool is used for Xenomai kernel. 
    Screenshot of latency test processing with a test duration of 60s.
     
  • After the Latency test successfully finishes, the program will generate the time series data diagram automatically in your test result output folder. You will see output similar to the screenshot below, which shows 10 minutes of data.
    Screenshot of latency datagram showing 10 minutes of data.

 

 

 

Cyclictest
  • Cyclictest tool is used for Preempt-RT kernel.
    Screenshot of Processing Cyclictest test
     

 

  • After the Cyclictest successfully finishes, the program will generate the time series data diagram automatically in your test result output folder. You will see output similar to the screenshot below, which shows 10 minutes of data.
    Screenshot of Cyclictest datagram showing 10 minutes of data.

 

Rhealstone 

Rhealstone is a measurement targeted specifically toward true multitasking solutions.  In this benchmark four categories of activity crucial to the performance of real-time systems represented. 

  • Task switching time 
  • Preemption time 
  • Semaphore shuffling time 
  • Deadlock breaking time 

The results obtained from Rhealstone benchmark are measured time to perform task switching, preemption, semaphore shuffling, and deadlock breaking. Lower values are better. 

Screenshot of Processing Rhealstone benchmark with 5 test cycles.

 

MSI-Latency 

MSI (Message Signaled Interrupt) Latency measures interrupt latency for MSIs generated by a discrete or integrated peripheral device, such as the Intel® Ethernet Controller I210 and serviced by the Intel® architecture core. This benchmark measures i210 kernel module MSI interrupt triggered back latency. 

NOTE: This benchmark only functions in tandem with an Intel® Ethernet Controller I210. 

Screenshot of Processing MSI Latency with a duration of 60s.
 
MSI-Jitter 

MSI (Message Signaled Interrupt) jitter benchmark tool is a benchmark tool to check the jitter of cyclical MSI from Intel® Ethernet Controller I210. This tool records each cyclical MSI timestamps in the software interrupt handler and calculates delta between each timestamps and compares the delta with cycle time as the jitter. 

NOTE: This benchmark only functions in tandem with an Intel® Ethernet Controller I210. 

Screenshot of Processing MSI Jitter test with duration of 60s.
 
MMIO-Latency

MMIO-latency is a simple driver that creates an affinitized thread to read a virtual map physical address (Memory-mapped I/O). Memory read latency is measured. The thread is created and started on init and loops LOOPS number of times. It also provides a char device that reads the current statistic counters, by using inline assembly and kernel function to get a very close benchmark.

 

Screenshot of MMIO-latency

 

 

Jitter

Screenshot of Jitter

The Jitter benchmark measures the execution time variation of a CPU test workload. The performance of the workload is impacted by kernel interrupts. Minimizing these interrupts also minimizes the jitter that applications could potentially experience. 

 

Caterpillar 

The Caterpillar benchmark measures the execution time variation of a memory test workload. The performance of the workload is impacted by cache misses. Using CAT improves application performance by assigning CPU affinity to cache ways, which can be dedicated to real-time applications.

Screenshot of Caterpillar

 

LMbench 

LMbench is a suite of simple, portable, ANSI/C microbenchmarks for UNIX/POSIX. In general, it measures two key features: latency and bandwidth. LMbench is intended to provide system developers an insight into basic costs of key operations.

Screenshot of LMbench

 

lat_mem_rd measures memory read latency for varying memory sizes and strides. The results are reported in nanoseconds per load and have been verified to be accurate to within a few nanoseconds on an SGI Indy. The entire memory hierarchy is measured, including onboard cache latency and size, external cache latency and size, main memory latency, and TLB miss latency. Only data accesses are measured; the instruction cache is not measured.

 

RTPM Test Configurations

For RTPM there are two ways to set up the running configurations. 

One is to use “Configuration File Parameters” and another one is use “Command Line Parameters”. 

The RTPM command line interface receives the “Command Line Parameters,” then translates them into “Configuration File Parameters,” so the “Command Line Parameters” has higher priority than “Configuration File Parameters”. The “Command Line Parameters” only control the benchmarks enable status. For more benchmark parameter settings, you need to use “Configuration File Parameters.”

Screenshot of Configuration File Parameters

Screenshot of Configuration File Parameters

 

 

Configuration File Parameters

The performance test can be configured in the esdq/RTPM_Test_Module/rt_performance_test/test_cfg file. 

  • Latency  
    • To execute Latency in RTPM:  
      • ENABLE_LATENCY=”TRUE”  
    • To NOT execute Latency or Cyclictest in RTPM: 
      • ENABLE_LATENCY=”FALSE”.
    • To set CPU core affinity (e.g., core 1): 
      • LATENCY_RUN_CORE=1 
    • To set Latency  test duration in seconds (e.g., 1 hour): 
      • LATENCY_TEST_DURATION=”3600” 
  • Cyclictest 
    • To execute Cyclictest in RTPM:   
      • ENABLE_CYCLICTEST=”TRUE”   
    • To NOT execute Cyclictest in RTPM:  
      • ENABLE_ CYCLICTEST =”FALSE”.  
    • To set CPU core affinity (e.g., core 1): 
      • CYCLICTEST_RUN_CORE=1 
    • To set Cyclictest test duration in seconds (e.g., 1 hour):  
      • CYCLICTEST_TEST_DURATION=”3600”
  • Rhealstone 
    • To execute Rhealstone in RTPM: 
      • ENABLE_RHEALSTONE=”TRUE” 
    • To NOT execute Rhealstone in RTPM: 
      • ENABLE_ RHEALSTONE =”FALSE”. 
    • To set CPU core affinity (e.g., core 1): 
      • RHEALSTONE_RUN_CORE=1
    • To set Rhealstone test cycles [0 to N] (e.g., 10000): 
      • RHEALSTONE_CYCLES=”10000” 
  • MSI Latency  
    • To execute MSI Latency in RTPM: 
      • ENABLE_MSI_LATENCY=”TRUE” 
    • To NOT execute MSI Latency in RTPM: 
      • ENABLE_MSI_LATENCY=”FALSE” 
    • To set the Intel® Ethernet Controller I210 ID: Use lspci command to find out the ethernet controller ID (e.g., 02:00.0). 
      • MSI_LATENCY_IGBID=”02:00.0” 
    • To set CPU core affinity (e.g., core 1):  
      • MSI_LATENCY_RUN_CORE=”1”  
    • Run IRQ as a pthread [0] or in legacy mode [1] (e.g., legacy mode): 
      • MSI_LATENCY_IRQ_SPEC=1 
    • Duration (clock ticks) for which the interrupts are blocked after requesting MSI (e.g., block 100 clock ticks): 
      • MSI_LATENCY_BLOCK_IRQ=100 
    • To set period between interrupts in milliseconds: 
      • MSI_LATENCY_IRQ_PERIOD=10 
    • To set the test duration in seconds (e.g., 1 hour): 
      • MSI_LATENCY_TEST_DURATION=3600 
  • MSI Jitter 
    • To execute MSI Jitter in RTPM: 
      • ENABLE_MSI_ JITTER =”TRUE” 
    • To NOT execute MSI Latency in RTPM: 
      • ENABLE_MSI_ JITTER =”FALSE” 
    • To set the Intel® Ethernet Controller I210 ID: Use lspci command to find out the ethernet controller ID (e.g., 02:00.0). 
      • MSI_ JITTER _IGBID=”02:00.0” 
    • To set CPU core affinity (e.g., core 1):  
      • MSI_ JITTER _RUN_CORE=”1”  
    • To set period between interrupts in milliseconds: 
      • MSI_ JITTER _IRQ_PERIOD=10 
    • To set the test duration in seconds (e.g., 1 hour): 
      • MSI_ JITTER _TEST_DURATION=3600 
  • MMIO Latency
    • To execute MMIO Latency in RTPM: 
      • ENABLE_MMIO_LATENCY=”TRUE” 
    • To NOT execute MMIO Latency in RTPM:
      • ENABLE_MMIO_LATENCY=”FALSE” 
    • To set the MMIO address (e.g., a123000). 
      • MMIO_LATENCY_PHYSICAL_ADDRESS=”a123000” 
    • To set CPU core affinity (e.g., core 1): 
      • MMIO_RUN_CORE=1 
    • To set if block the IRQs respectively (e.g., 1 or 0, default 0 is to not block IRQs.). 
      • MMIO_LATENCY_IRQ_STATE=0 
    • To set the test duration in seconds (e.g., 5 mins): 
      • MMIO_LATENCY_DURATION=300
  • Jitter 

    • To execute Jitter in RTPM:  
      • ENABLE_JITTER_TEST=”TRUE”  
    • To NOT execute Jitter in RTPM  
      • ENABLE_JITTER_TEST =”FALSE”  
    • To set CPU core affinity (e.g., core 1): 
      • JITTER_RUN_CORE=1 
    • To set display update rate (e.g., 20000): 
      • JITTER_DISPLAY_UPDATE_RATE=20000 
    • To set loop count for code block (e.g., 80000): 
      • JITTER_LOOP_COUNT=80000 
    • To set sample counts after which program terminates (e.g., 200) 
      • JITTER_SAMPLE_COUNTS=200 
  • Caterpillar 
    • To execute Caterpillar in RTPM:  
      • ENABLE_CATERPILLAR=”TRUE”  
    • To NOT execute Caterpillar in RTPM  
      • ENABLE_CATERPILLAR=”FALSE”  
    • To set CPU core affinity (e.g., core 1): 
      • CATERPILLAR_RUN_CORE=1 
    • To set number of test cycles to complete per sample (e.g., 10000): 
      • CATERPILLAR_TEST_CYCLES=10000 
    • To set number of iterations to execute operation (e.g., 50000): 
      • CATERPILLAR_TEST_ITERATIONS=50000 
    • To set maximum number of samples to take (e.g., 200) 
      • CATERPILLAR_MAX_SAMPLES=200 
  • LMbench 
    • To execute LMbench in RTPM:  
      • ENABLE_LMBENCH=”TRUE”  
    • To NOT execute LMbench in RTPM  
      • ENABLE_ LMBENCH =”FALSE”  
    • To set CPU core affinity (e.g., core 1): 
      • LMBENCH_RUN_CORE=1 
    • To set test memory size in megabytes (e.g., 128): 
      • LMBENCH_TEST_MEMORY_SIZE=128
  • Stress: Stress-ng is used to simulate non-real-time workload in the target system during the performance test.  
    • To enable stress-ng: 
      • ENABLE_STRESS=”TRUE” 
    • To disable stress-ng: 
      • ENABLE_STRESS=”FALSE” 
    • To set up “stress” workload core affinity (e.g., core 0): 
      • STRESS_CORE_SETUP_STRESS=0 
    • To set up “stress-ng” workload core affinity (e.g., core 2): 
      • STRESS_CORE_SETUP_STRESS_NG=2 
    • To set up “hackbench” workload core affinity (e.g., core 3): 
      • STRESS_CORE_SETUP_HACKBENCH=3 
    • To set the timeout of stress workload after test complete in seconds (e.g., 3 seconds): 
      • STRESS_TIMEOUT=3 
  • RTPM main progress setup. 
    • To set up the “RTPM” main progress core affinity (e.g., core 1): 
      • RTPM_MAIN_CORE=1 

 

Command Line Parameters 

RTPM test module can be executed separately. It supports the command line parameters below: 

-p or --logpath

Specify log file path (required parameter). 

-n or --logname 

Specify csv log file name (required parameter). 

-l or --latency

Execute Latency  test. If not specified, ENABLE_LATENCY setting in configuration file will be used. 

-c or --cyclictest

Execute Cyclictest test. If not specified, ENABLE_CYCLICTEST setting in configuration file will be used.

-r   or --rhealstone

Execute Rhealstone test. If not specified, ENABLE_RHEALSTONE setting in configuration file will be used. 

-ml or --msilatency

Execute MSI Latency test. If not specified, ENABLE_MSI_LATENCY setting in configuration file will be used. 

-mj or --msijitter

Execute MSI Jitter test. If not specified, ENABLE_MSI_JITTER setting in configuration file will be used. 

-mo or --mmiolatency

Execute MMIO-Latency test. If not specified, ENABLE_MMIO_LATENCY setting in configuration file will be used.  

-cp or --caterpillar

Execute Caterpillar test. If not specified, ENABLE_CATERPILLAR setting in configuration file will be used.   

-j or --jitter

Execute Jitter test. If not specified, ENABLE_JITTER_TEST setting in configuration file will be used.   

-lm or --lmbench

Execute LMbench test. If not specified, ENABLE_LMBENCH setting in configuration file will be used.

-s or --stress

Add stress during performance test. If not specified, ENABLE_STRESS setting in configuration file will be used. 

-a or --all

Execute all performance tests, except Latency and Cyclictest. The program will identify the current Real-Time framework. If it is Xenomai, it will run Latency, otherwise if it is Preempt-RT, it will run Cyclictest.

If not specified, settings in configuration file will be used. 

-h or --help

Show help. 

Parameter Mapping Table 

The table helps explain the same impact as the configuration file. When the command line takes effect, if the “Command Line Parameter” is not given, RTPM will use “Configuration File Parameter” as the default setup. 

Command Line Parameter  Configuration File Parameter 
-l or --latency  ENABLE_LATENCY=”TRUE” 
-c or --cyclictest  ENABLE_CYCLICTEST=”TRUE” 
-r or --rhealstone  ENABLE_RHEALSTONE=”TRUE” 
-ml or --msilatency  ENABLE_MSI_LATENCY=”TRUE” 
-mj or --msijitter ENABLE_MSI_JITTER=”TRUE” 
-mo or --mmiolatency ENABLE_MMIO_LATENCY=”TRUE” 
-cp or --caterpillar  ENABLE_CATERPILLAR=”TRUE” 
-j or --jitter  ENABLE_JITTER_TEST=”TRUE” 
-lm or --lmbench  ENABLE_LMBENCH=”TRUE” 
-a or --all 

ENABLE_LATENCY=”TRUE” (For Xenomai only)

ENABLE_CYCLICTEST=”TRUE”  (For Preempt-RT only)

ENABLE_RHEALSTONE=”TRUE” 

ENABLE_MSI_LATENCY=”TRUE” 

ENABLE_MSI_JITTER=”TRUE” 

ENABLE_MMIO_LATENCY=”TRUE” 

ENABLE_CATERPILLAR=”TRUE” 

ENABLE_JITTER_TEST=”TRUE” 

ENABLE_LMBENCH=”TRUE” 

 

 

Time-Sensitive Networking (TSN) Test Modules

 

TSN Performance Measurement

Time-sensitive networking (TSN) is set to reshape the industrial communication landscape and lay the foundation for the convergence of Information Technology (IT) and Industrial Operations Technology (OT). By bringing industrial-grade robustness and reliability to Ethernet, TSN offers an IEEE standard communication technology that enables interoperability between standard-conformant industrial devices from any vendor.

On a time-sensitive network system, there are several factors that could impact the deterministic of real-time data streams. These factors include hardware design, BIOS configuration, OS kernel configuration, system settings, etc. TSN performance measurement module is designed to check the key settings of the system and help you to identify hotspots of the system for real-time network performance and give a recommendation based on Best-Known Configuration (BKC). In addition, it provides a way to measure the real-time data transmission performance with some open source tools.

The figure shows the hardware setup for the TSN performance test. Each platform contains an integrated or discrete Ethernet Controller with TSN support from Intel.

For this test, assign two types of platforms as shown in the table below. Connect a CAT-5E Ethernet cable to the RJ45 connector of Talker and Listener.

Talker The platform periodically sends the packet via a TSN stream. 
Listener  The platform continuously receives the packet via a TSN stream. 

 

ESDQ ECI TSN Hardware Setup

 

The TSN performance measurement module interacts with the Intel® ESDQ CLI through a common test module interface layer (TMI) which is part of the Intel® ESDQ binary.
Intel® ESDQ generates a complete test report in HTML format, along with detailed logs packaged as one zip file, which you can manually choose to email to the ESH support team.

TSN contains the following test modules:

Real-Time Readiness Check

Screenshot of Real-Time readiness Check

Screenshot of Real-Time Readiness Check

This module leverages one of the tools in Intel® Time Coordinated Computing Tools (Intel® TCC Tools) to check the many attributes that may affect time-sensitive network performance.

  • Verifies whether the system has a supported processor, BIOS, and OS. 
  • Checks for features that may affect real-time performance, such as Intel® Turbo Boost Technology, Enhanced Intel SpeedStep® Technology, and processor power-saving states. 
  • Reports CPU and GPU frequencies. 
  • Operates at the OS level.


Boot Command Line Check

This module checks the Real-Time OS boot command line parameters which may affect time-sensitive network performance and gives recommended settings per best-known configurations.

Screenshot of TSN Boot Command Line Check

 

Kernel Configuration Check

This module checks the Real-Time OS kernel configuration that may affect time-sensitive network performance and gives recommendation settings per best-known configurations.

Screenshot of TSN Kernel Config Check

 

TSN Performance Test 

This module contains the below benchmarks to evaluate the performance of the time-sensitivity network.  


Packet Latency Measurement

This module leverages one of the tools in the project IOTG TSN Reference Software for Linux to bounded low-latency transmission/reception achievable via AF_PACKET standard Linux socket interfaces while leveraging device-specific Ethernet-TSN features.

In scheduled traffic, all frames are transmitted and received periodically. In this benchmark, the performance indicator for scheduled traffic is to measure its inter-packet latency distribution and end-to-end latency distribution over an extended period. The closer the inter-packet measured value is to the cycle value, the better the performance. The smaller the end-to-end measured value, the better the performance. (In QBV_ETF mode, the lead time of packet transmission is configured as 100us in this benchmark.) 

Screenshot of TSN Inter-Packet Latency check

 

NOTE: The performance result above only reported on “listener”.

 

TSN Test Configurations

Configuration File Parameters

The performance test parameters can be configured in the esdq/Tsn_Test_Module/tsn_performance_test/test_cfg file.

Screenshot of TSN Config File Parameters

 

  • The type of the device 
    • The device periodically sends the packet via a TSN stream. NODE_TYPE should be set as “talker”:  
      • NODE_TYPE=”talker”  
    • The device continuously receives the packet via a TSN stream. NODE_TYPE should be set as “listener”: 
      • NODE_TYPE=”listener” 
  • The name of the ethernet interface 
    • This is the specific ethernet interface name used in your device (e.g. “enp4s0”): 
      • NET_INTERFACE=”enp4s0” 
  • The number of packets used in this measurement 
    • Default value is 100000, value range from 10 to 10000000. 
      • PACKET_NUMBER=100000 
  • The type of traffic scheduler mode  
    • Default value is QBV_ETF, available values are “QBV_ETF” and “QBV”.

      • TSN_MODE=”QBV_ETF”

Command Line Parameters

TSN test module supports the command line parameters below: 
 

-p or --logpath

Specify log file path (required parameter). 
 

-n or --logname

Specify csv log file name (required parameter). 

 

-i or --interface

Specify the Ethernet interface name used in this case. 

 

-t or --type

Specify the device type (“talker” or “listener”) taken in this case.

 

-h or --help

Show help. 

 

To run TSN test module in command line with the specified parameters, perform the following:

  1. [Talker and Listener] Change directory to esdq: 
    cd $HOME/edge_software_device_qualification/Edge_Software_Device_Qualification_For_RTPM_<version>/esdq/
    Where <version> is the package version that was downloaded.
     
  2. [Talker and Listener] Get TSN test module ID <TSN_MODULE_ID> using the command:
    ./esdq list
    
    +------------------+---------+--------------------------+
    |   Test Module    | Version |            ID            |
    +------------------+---------+--------------------------+
    | TSN_Test_Module  |   1.6   | xxxxxxxxxxxxxxxxxxxxxxxx |
    | Rtpm_Test_Module |   1.9   | xxxxxxxxxxxxxxxxxxxxxxxx |
    +------------------+---------+--------------------------+ 

     

  3.  [Listener] Run TSN test module using the command:

    ./esdq run -r <TSN_MODULE_ID> -p "-i $IFACE -t listener" 

    Substitute $IFACE with the mounted interface corresponding to the integrated or discrete Ethernet Controller with TSN support from Intel.

  4. [Talker] Run TSN test module using the command:

    ./esdq run -r <TSN_MODULE_ID> -p "-i $IFACE -t talker" 

    Substitute $IFACE with the mounted interface corresponding to the integrated or discrete Ethernet Controller with TSN support from Intel.

 

 


Get Started 

Intel® ESDQ 6.0 CLI tool is installed as part of the Intel® ESDQ for Intel® ECI. 

NOTE: Be aware that screenshots may show a package version number that is different from the current release. See the Release Notes for information on the current release. 

Download and Install Intel® ESDQ for Intel® ECI 

  1. Select Configure & Download to download Intel® ESDQ ECI and then follow the steps below to install it. 
    Configure & Download
     
  2. Select Request Access
    Download Intel ESDQ Intel ECI
     
  3. After access approval, the configure screen appears as shown below. Select Version or TagTarget System, and Distribution on the screen and then click Download
    Download Recommended Configuration
     
  4. Copy and Save the Product Key
    Copy product key
     
  5. Click the Edit button to reconfigure the options and download another version of ESDQ for ECI in subsequent releases.
  6. Transfer the downloaded Intel® ECI package to the target Debian system.
  7. View the edgesoftware_configuration.xml included in the edge software zip package and look for the Intel® ESDQ ingredient and test modules for Intel® ECI. 
     <project path="installation/src" id="61421ecad393cc0020ed22d7" version="1.5" label="esdq/Rtpm_Test_Module"/>
  8. Prepare the target system for Intel® ECI installation: 
    • Make sure the target system is Debian 11 Linux.
    • Make sure apt source list is updated to latest and sudo, lsusb, and unzip are already installed. 
      • sudo apt update 
      • sudo apt install usbutils unzip python3-setuptools python3-psutil gnuplot
    • Make sure Python-3 environment is set up correctly.
      • export LC_ALL=C.UTF-8 
      • export LANG=C.UTF-8  
    • Make sure all RTPM dependent modules are installed. 
      • For Xenomai patched Debian 11 Linux. 
        • apt install rtpm-xenomai 
      • For Preempt-RT patched Debian 11 Linux. 
        • apt install rtpm 
    • Make sure the target system date is correct 
      • Use “date” command to validate current system date. 
    • Make sure eci-process-automation package is installed for TSN testing. 
      • sudo apt install eci-process-automation
  9. Provide executable permission to edgesoftware
    chmod +x ./edgesoftware
  10. Run the command below to install Intel® ESDQ 6.0 and Intel® ECI. 
    ./edgesoftware install
  11. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download (or from the saved Product Key in the step above). 
  12. When the installation is complete, you see the message “Installation of package complete” and the installation status for each module. 
    Screenshot of Installation of Package Complete
     
  13. Find the module ID for the Intel® ESDQ by running the command: 
    ./edgesoftware list --default

    Example output:  
    Screenshot of edgesoftware list command
     
  14. After modules installation is completed successfully, reboot the system:
    reboot

Run the Application

For the complete Intel® ESDQ CLI, refer to Intel® ESDQ CLI Overview.  The test modules are already available in the target system.

 

Single DUT Setup for RTPM

 

  1. Change directory to RTPM: 
    cd $HOME/edge_software_device_qualification/Edge_Software_Device_Qualification_For_RTPM_<version>/esdq
    Where <version> is the package version that was downloaded.
  2. Get RTPM test module ID with the command:
    ./esdq list
    
    +------------------+---------+--------------------------+
    |   Test Module    | Version |            ID            |
    +------------------+---------+--------------------------+
    | TSN_Test_Module  |   1.6   | xxxxxxxxxxxxxxxxxxxxxxxx |
    | Rtpm_Test_Module |   1.9   | xxxxxxxxxxxxxxxxxxxxxxxx |
    +------------------+---------+--------------------------+
    
  3. Use ESDQ command parameters “-p” to pass the arguments to RTPM test module: 
    ./esdq run < Rtpm_Test_Module_ID> -r -p “-a”
  4. These are three situations that need attention: 
    1. Pass a blank parameter by ESDQ “-p” command like below: 
       ./esdq run < Rtpm_Test_Module_ID> -r -p “”
      The RTPM will not receive any parameters and will use test_cfg setup to configure whole test progress. 
    2. Pass invalid parameters by ESDQ “-p” command like below: 
      ./esdq run < Rtpm_Test_Module_ID> -r -p “invalid parameters”
      The invalid parameter will be filtered and replace it with run all benchmarks, thus RTPM will run all benchmarks. 
    3. Skip esdq “-p” command and run like below: 
      ./esdq run < Rtpm_Test_Module_ID> -r
      The RTPM will run all benchmarks. 

      For more ESDQ command line parameters information, refer to Intel® Edge Software Device Qualification (Intel® ESDQ) Overview.

The test report should look like the following example:

Note: The results shown below are for illustration only.

Summary of test report

Test Report Modules Info

 

Test Report Test Suites

Test Report Test Suites part 2

Screenshot of Report

If the Latency or Cyclictest benchmark test finished without any issue, then the program will generate the time series data diagrams and integrate them into the HTML report similar to the output below:

Screenshot showing Latency and Cyclictest datagrams

 

 

Dual DUT Setup for TSN

 

  1. [Talker and Listener] Change directory to esdq: 
    cd $HOME/edge_software_device_qualification/Edge_Software_Device_Qualification_For_RTPM_<version>/esdq/
    Where <version> is the package version that was downloaded.
     
  2. [Talker and Listener] Get TSN test module ID <TSN_MODULE_ID> using the command:
    ./esdq list
    
    +------------------+---------+--------------------------+
    |   Test Module    | Version |            ID            |
    +------------------+---------+--------------------------+
    | TSN_Test_Module  |   1.6   | xxxxxxxxxxxxxxxxxxxxxxxx |
    | Rtpm_Test_Module |   1.9   | xxxxxxxxxxxxxxxxxxxxxxxx |
    +------------------+---------+--------------------------+ 

     

  3.  [Listener] Change the parameters in the configure file. Specify the Ethernet interface name and device type used in your setup:

    vi Tsn_Test_Module/tsn_performance_test/test_cfg
    
       NODE_TYPE="listener" 
       NET_INTERFACE="xxxxxx" 
    

     

  4. [Talker] Change the parameters in the configure file. Specify the Ethernet interface name and device type used in your setup:

    vi Tsn_Test_Module/tsn_performance_test/test_cfg
    
       NODE_TYPE="talker" 
       NET_INTERFACE="xxxxxx" 

     

  5. [Listener] Run Intel® ESDQ TSN test and generate report:

    ./esdq run -r <TSN_MODULE_ID>

     

  6. [Talker] Run Intel® ESDQ TSN test and generate report:

    ./esdq run -r <TSN_MODULE_ID>

 

The test report should look like the following example on “listener”:
Note: The results shown below are for illustration only.

TSN Test Report 1

TSN Test Report 2

TSN Test Report 3

 

 


Release Notes 

RTPM Release Notes

Current Version 1.9 (Package Version 2022.3)

New in this Release 
  • This version has been upgraded for compatibility with ECI 3.0.  
  • If you plan to use the latest ECI 3.0, use it with Intel ESDQ ECI version 2022.3.

 

Version 1.8    

  • Leveraged gnuplot function to generate the Latency and Cyclictest test data diagram.
  • Changed the logic for the command line argument “-a”. When using the “-a” command, the RTPM will choose the Latency or Cyclictest benchmark according to system real-time framework instead of running them. For example, if system is Xenomai, the RTPM will choose Latency; if system is Preempt-RT, the RTPM will choose Cyclictest. You can also choose to use the separated command “-c -l” to run them in one test flow.
  • Separated stress workload core affinity setup, you can configure each “stress” workload core affinity in test_cfg now.

 

Version 1.7

  • Added the below Real-time Performance test. 
    • Jitter 
    • Caterpillar 
    • LMbench 
  • Separated Latency and Cyclictest test flow, and made Latency & Cyclictest run independently. 
  • Added more benchmark running information. 
  • Automatically compressed test result for archiving.

Version 1.6 

  • Support Real-Time Readiness Checker 1.3. 
  • Add below Real-time Performance test.  
    • MMIO-Latency

Version: 1.5 

  • Replace BIOS configuration check with: 
    • Real-Time Readiness Checker. 
  • Add below Real-time Performance test. 
    • Rhealstone 
    • MSI-latency 
    • MSI-Jitter

Version: 1.0

Initial features for recommended configuration.

 

Known Issues 

  • Some BIOS settings may be “LOCKED” or detected incorrectly in OS. Check BIOS manually for the recommended settings. 

 

TSN Release Notes

Current Version: 1.6

New in this Release   
  • Remove OPC UA performance measurement case. 
  • Supports Intel Atom® x6000E Series Processors.

Version 1.5

  • Add new measurement of packet end-to-end transmission latency.

Version 1.2   

  • Supports Intel® Ethernet Controller I225-IT for Time-Sensitive Networking (TSN). 
  • Added performance measurement based on OPC UA stack.

Version 1.0

  • Initial features for recommended configuration.

 


Support Forum

If you're unable to resolve your issues, contact the Support Forum

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.