Developer Guide

  • 2021.1
  • 11/03/2021
  • Public
Contents

Step 3: Preproduction: Generate a Tuning Config

In this step, you will walk through the data streams optimizer preproduction workflow to generate a tuning configuration that meets the RTCP latency requirement along with power consumption optimization. The demo focuses on tuning the PCIe-from-memory stream and power consumption.
This demo does not measure power consumption, but power consumption can be measured using tools outside of Intel® TCC Tools, such as Intel® SoC Watch. For 11th Gen Intel® Core™ processors, you can find a compatible version of Intel® SoC Watch in Intel® System Studio 2020 Update 3. For Intel Atom® x6000E Series processors, ask your Intel representative for access to the appropriate version of Intel® SoC Watch.
The data streams optimizer requires the following input files: environment file, requirements file, and workload validation script. For this demo, you will use provided samples of these files.
Example output shown here is for illustration only. Your output may vary.

Preproduction Workflow

These steps assume a host-target environment.
  1. Confirm that RTCM is still disabled on the target board:
    1. From your host system, connect to the target system:
      ssh <user>@<target>
    2. In the SSH session
      , run the following command to confirm that RTCM is still disabled:
      /usr/share/tcc_tools/scripts/setup_ssram/control_rtcm.sh status
      If RTCM is enabled, disable RTCM with the following commands:
      /usr/share/tcc_tools/scripts/setup_ssram/control_rtcm.sh disable reboot
  2. On the host system
    , source the environment file to set up environment variables:
    source ~/intel/oneapi/setvars.sh
  3. Go to the
    tools
    directory:
    cd ${TCC_TOOLS_PATH}
  4. Review the sample environment file:
    1. Open the sample environment file. This example command uses nano, but you can use any text editor.
      nano ./demo/environment/sample_environment.json
    2. Modify the following fields (see example file below):
      Field Name
      Description
      "hostname"
      Replace
      ...
      with the IP address or host name of the target system.
      "username"
      Replace
      ...
      with
      root
      .
      The field should look like this:
      "username": "root"
      This field contains the username to connect to the target board by SSH.
      "password"
      Replace
      ...
      with the password to connect to the target board by SSH.
      If the SSH connection is expected to be passwordless, leave the password field empty as shown below:
      "password": ""
      All other fields
      Leave all other fields as is. These fields enable you to further customize the behavior of the tool, but customization is beyond the scope of this demo.
      Environment file example (“targets” section only):
      "targets": { "target_name_1": { "target_info": "...", "reboot_command": "reboot", "connections_settings": { "hostname": "...", "username": "root", "password": "", "port": 22, "connection_timeout":5, "reconnection_timeout":10, "reconnection_attempts":10 },
    3. Save and close the file.
  5. Review the sample requirements file:
    1. Open the sample requirements file. This example command uses nano, but you can use any text editor.
      nano ./demo/requirements/single_pciememrd_0.json
    2. For your reference, note the
      "command"
      field. This is the same script you used in the previous step, Step 2: Run RTCP on Untuned System. The tool will run the script to validate whether the tuning configuration meets the RTCP latency requirement.
    3. In the
      "arguments"
      field, replace
      <target_pcie_address>
      and
      <packet_generator_pcie_address>
      per the table below. Make sure to delete the angle brackets (< >) too.
      Argument
      Description
      --pci_rtcp <target_pcie_address>
      Specify the address of the PCIe device on the target board.
      Format: 0000:<address>.
      Example: If the address is 01:00.0, you would enter
      0000:01:00.0
      in this argument.
      --pci_pglm <packet_generator_pcie_address>
      Specify the address of the PCIe device on the packet generator board.
      Format: 0000:<address>.
      Example: If the address is aa:00.0, you would enter
      0000:aa:00.0
      in this argument.
      Note:
      To get the PCIe device address, run the following command on each board:
      lspci | grep I210
      The following example output shows the address is aa:00.0. In this case, you would enter
      0000:aa:00.0
      in the appropriate argument.
      aa:00.0 Ethernet controller: Intel® Corporation I210 Gigabit Network Connection (rev 03)
    4. Verify that the
      "consumer"
      field is set in the
      "requirements"
      section and contains the address of the PCIe device on the target board.
      Example: If the address is 01:00.0, you would enter
      01:00.0
      in the
      "consumer"
      argument.
    5. Save and close the file.
  6. On the host system, run the preproduction tool to search for a tuning configuration:
    python3 tcc_data_streams_optimizer_preprod.py search -e ./demo/environment/sample_environment.json -r ./demo/requirements/single_pciememrd_0.json
    For your reference, the following table contains a description of each argument.
    Option
    Description
    -e
    Path to the sample environment file.
    -r
    Path to the sample requirements file.
  7. Confirm that you see output similar to the example below. The output shows that the tool first checks for dependencies, such as input files and ability to connect to the target via SSH.
    Processing environment file: ./demo/environment/sample_environment.json ... Environment file parsed. Processing requirement file: ./demo/requirements/single_pciememrd_0.json ... Requirement file parsed. Creating output folder: /home/<user>/intel/oneapi/tcc_tools/latest/tools/<target_hostname>/single_pciememrd_0_<date> Connecting to <target_hostname>... Connected.
    The tool finds the first suitable tuning configuration.
    The tool prints a list of messages to the log file. The messages describe the affected settings. The level of detail in these messages balances the need to provide useful information vs. the need to protect Intel proprietary information. For more information about each message, see Tuning Configurations.
    The tool generates a capsule (binary) of the configuration. The capsule is used to apply the configuration to the target. The target reboots.
    Connection to database tuning_tgl.db - successful. Searching for suitable tuning configuration... Stream Memory -> 0:1c:0:0: configuration 1 out of 8 Tuning binary has been signed. Generating the capsule of the configuration to tune the system - for the settings of this configuration see single_pciememrd_0.log in the output folder. Capsule was generated. Moving capsule from ./tmp/streams_capsule.out.bin on Host to /tmp/streams_capsule.out.bin on <target_hostname>. Capsule was moved. Capsule was applied for <target_hostname>.
  8. Wait for the target board to reboot. It may take 1 minute or even more. While the target is rebooting, the tool will attempt to connect to the target repeatedly based on the
    <reconnection_attempts>
    field in the environment file. Example output:
    Rebooting <target_hostname> Attempt 1. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 2. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 3. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 4. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 5. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 6. Reconnecting to <target_hostname>... Connected.
    After reconnecting to the target, the tool runs the workload validation script. To demonstrate the iterative process of tuning, the latency requirement for this demo was chosen such that the first tuning iteration optimizes the power consumption but exceeds the latency requirement:
    Validating configuration for target <target_hostname>... Starting validation script: python3 /usr/share/tcc_tools/tools/demo/workloads/bin/rtcp_validation_script.py. Validation script output: ====================================================== Return code: 1 stdout Start validation Validation is finished. Please wait for results processing. Validation information: iterations: 1985353 duration: 120 sec cpuid: 3 stress: No Required value less than 65.0 us. Statistic: Min |Max |Avg -------------------------------------- 10.72 us |98.872 us |19.0 us ====================================== Deadline |Iterations |Passed |Failed --------------------------------------------------- 65.0 us |1985353 |1982946 |2407 =================================================== Max latency is too big Fail ====================================================== For finding more details look at: /home/test/intel/oneapi/tcc_tools/latest/tools/<target_hostname>/single_pciememrd_0_<date>/single_pciememrd_0.log Validation script FAILED
    If you see
    VALIDATION ERROR
    with a path to a log, this log is located on your target board.
    When the validation script fails, the tool repeats the tuning flow. It finds another suitable tuning configuration or exits if none are found. In this case, the tool finds another configuration:
    Searching for suitable tuning configuration... Stream Memory -> 0:1c:0:0: configuration 2 out of 8 Tuning binary has been signed. Generating the capsule of the configuration to tune the system - for the settings of this configuration see single_pciememrd_0.log in the output folder. Capsule was generated. Moving capsule from ./tmp/streams_capsule.out.bin on Host to /tmp/streams_capsule.out.bin on <target_hostname>. Capsule was moved. Applying streams capsule for <target_hostname> ...
  9. Wait for the target board to reboot. It may take 1 minute or even more. While the target is rebooting, the tool will attempt to connect to the target repeatedly based on the
    <reconnection_attempts>
    field in the environment file. Example output:
    Rebooting <target_hostname> Attempt 1. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 2. Reconnecting to <target_hostname>... Connection attempt failed. Trying again... Attempt 3. Reconnecting to <target_hostname>... Connected.
    After reconnecting to the target, the tool runs the workload validation script. Now the script shows that the maximum latency measurement meets the deadline.
    Validating configuration for target <target_hostname>... Starting validation script: /usr/share/tcc_tools/tools/demo/workloads/bin/rtcp_validation_script.py Validation script output: ====================================================== Return code: 0 stdout Run noisy neighbor Start validation Stop noisy neighbor Validation is finished. Please wait for results processing. Validation information: iterations: 4720191 duration: 120 sec cpuid: 3 stress: No Required value less than 65.0 us. Statistic: Min |Max |Avg -------------------------------------- 8.328 us |36.368 us |11.0 us ====================================== Deadline |Iterations |Passed |Failed --------------------------------------------------- 65.0 us |4720191 |4720191 |0 =================================================== Success ====================================================== Validation script PASSED
    After the validation passes, the tool generates a tuning configuration file and exits. Example output:
    Configuration for Target for session: <user>@<target_hostname> found. Creating tuning configuration... See /home/<user>/intel/oneapi/tcc_tools/latest/tools/<target_hostname>/single_pciememrd_0_<date>/single_pciememrd_0.tuning_configuration.json for configuration details. Tuning configuration was created. Path to output file: /home/<user>/intel/oneapi/tcc_tools/latest/tools/<target_hostname>/single_pciememrd_0_<date>/single_pciememrd_0.tuning_configuration.json For more information, see: /home/<user>/intel/oneapi/tcc_tools/latest/tools/<target_hostname>/single_pciememrd_0_<date>/single_pciememrd_0.log Bin file was moved to output. Capsule file was moved to output. Application exit
  10. After the application exits, confirm that the tool generated the tuning configuration file,
    single_pciememrd_0.tuning_configuration.json
    , in the following directory:
    cd <target_hostname>/single_pciememrd_0_<date> ls -la
In this demo, both the untuned system (step 2) and the tuned system (step 3) met the deadline. The data streams optimizer selected a configuration that balances latency and power requirements. This “power-friendly” configuration did not turn off as many power management settings as the untuned system with Intel® TCC Mode enabled only. The data streams optimizer achieved higher latency and lower power consumption compared to Intel® TCC Mode enabled. In a real-world use case, you can perform additional analysis outside of Intel® TCC Tools to determine if your system requirements, like power consumption, are met.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.