Developer Guide

  • 2021.1
  • 11/03/2021
  • Public
Contents

Control Data Collection

A collector library performs data collection and processing. The ITT APIs defined in
ittnotify.h
load a collector library during runtime, based on the value of the
INTEL_LIBITTNOTIFY64
environment variable. The measurement library includes a collector (
libtcc_collector.so
) to perform data collection and processing for real-time applications.
You can set the
INTEL_LIBITTNOTIFY64
environment variable as follows:
export INTEL_LIBITTNOTIFY64=libtcc_collector.so <additional environment variables to control data collection>
At the end of the command, you can add the following environment variables to control collection and processing:
Environment Variable
Description / Examples
TCC_USE_SHARED_MEMORY
Use a shared memory ring buffer instead of a local buffer to store all measurement results. Can be true or false. Default: false.
When the shared memory mode is enabled, the second (monitoring) application can attach to the shared buffer and process the measurement results.
TCC_MEASUREMENTS_BUFFERS
Define the buffers that will hold the measurements generated from your application. If your application has multiple measurement instances, you can define a buffer for each instance. One buffer will be created per instance.
Full format description:
[<measurement_name>:<buffer_size>[:deadline]]
[,<measurement_name>:<buffer_size>[:deadline]]…
For each measurement instance in your application, specify:
  • Measurement name: The measurement name must be the same as the one specified in the
    __itt_string_handle_create
    call in your application. See Instrument the Code.
  • Buffer size in number of measurements: The maximum number of measurements that can be stored for this measurement instance. There is a difference in how the buffer overflow is handled, depending on the
    TCC_USE_SHARED_MEMORY
    option. For non-shared mode, when the buffer is full, newly arriving data will always overwrite the existing values, starting from the oldest (ring buffer). For shared mode, new data will overwrite the oldest only after the oldest data has been read by the second (monitoring) application; otherwise, new data will be skipped.
    When choosing a buffer size for your use case, consider a balance between the amount of data that you need to analyze and the memory consumption. Choosing a large buffer will provide more data for analysis, but also consume more application memory. For long measurements with billions of iterations, shared memory is the recommended approach, as it usually consumes significantly less memory. In this case, you need to balance the writing speed of the real-time workload and the reading speed of the monitoring application to determine the optimal buffer size to avoid loss of measurements.
    The content of each buffer is stored in the
    TCC_MEASUREMENTS_DUMP_FILE
    when the program finishes. If buffer overflow occurred, only the last
    buffer_size
    measurements will be stored.
  • (Optional) Deadline: The deadline is the maximum tolerable latency. The deadline value does not influence collector behavior. You can decide how to use the value in your code. To specify the time unit of the deadline, use the TCC_MEASUREMENTS_TIME_UNIT environment variable. Default: CPU clock cycles.
Examples:
  • One measurement instance without deadline:
    TCC_MEASUREMENTS_BUFFERS=Workload1:1000
  • One measurement instance with deadline:
    TCC_MEASUREMENTS_BUFFERS=Workload1:1000:10000
  • Two measurement instances with deadline:
    TCC_MEASUREMENTS_BUFFERS=Workload1:1000:65535,Workload2:2000:100000
  • Three measurement instances with or without deadline:
    TCC_MEASUREMENTS_BUFFERS=Workload1:1000:1000,Workload2:100,Workload3:100:5000
TCC_MEASUREMENTS_DUMP_FILE
File where measurements are logged after the program finishes. If the measurement library cannot open the provided file, nothing will be stored.
NOTE:
Use either implicit output (TCC_MEASUREMENTS_DUMP_FILE) or explicit output (“print” described in Analyze Measurements Offline).
TCC_MEASUREMENTS_TIME_UNIT
Time unit that will be used to store the results. If the TCC measurement library cannot parse
TCC_MEASUREMENTS_TIME_UNIT
, clk will be used. Options:
  • clk: CPU clock cycles
  • ns: nanoseconds
  • us: microseconds
The deadline value in
TCC_MEASUREMENTS_BUFFERS
is optional and is not used by the measurement library directly. Instead, you can query the value using
tcc_measurement_get_deadline_from_env()
and then set this deadline value when calling
tcc_measurement_set_deadline()
.
Example of reading the deadline value from the environment:
/* Extract the deadline settings from the environment variable */ uint64_t deadline, deadline_raw; TCC_TIME_UNIT unit = tcc_measurement_get_time_unit_from_env(); deadline_raw = tcc_measurement_get_deadline_from_env(measurement_name); deadline = tcc_measurement_convert_time_units_to_clock(deadline_raw, unit);
If you use deadline handling in your app with different deadline values, supply the deadline through the environment to change the deadline value. This will simplify your code without needing to recompile your code and/or creating an app command line with a special option for deadline value.

Examples of Controlling the Collector Behavior

The following examples show the behavior of the measurement library collector depending on the environment variables. The examples assume that you have instrumented your real-time application
myapp
with ITT APIs, and the application has two measurement instances: “Input” and “Compute.”
  • Example 1: The measurement library collector will create a local ring buffer that can hold 1000 measurements. The collector will store the measurement results for the “Input” measurement instance in the buffer during application run. After the application is finished, the results will vanish.
    INTEL_LIBITTNOTIFY64=libtcc_collector.so TCC_MEASUREMENTS_BUFFERS=Input:1000 ./myapp
  • Example 2: The measurement library collector will create two buffers that can hold 1000 measurements each. The collector will store the measurement results in the buffers during application run. After the application is finished, the last 1000 results of each measurement instance will be stored in the dump.txt file. The format is explained in Analyze Measurements Offline. The data will be stored in CPU clock cycles.
    INTEL_LIBITTNOTIFY64=libtcc_collector.so TCC_MEASUREMENTS_DUMP_FILE=dump.txt TCC_MEASUREMENTS_BUFFERS=Input:1000,Compute:1000 ./myapp
  • Example 3: The measurement library collector will create two buffers in shared memory. Each buffer can hold 10 measurements each. The collector will store the measurement results in the buffers during application run. After the application is finished, the last 10 results of each measurement instance will be stored in the dump.txt file. The shared memory always keeps data in CPU clock cycles (raw measurements) while the output file (dump.txt) uses the configured time units. The deadline value does not influence collector behavior. You can decide how to use the value in your code.
    INTEL_LIBITTNOTIFY64=libtcc_collector.so TCC_USE_SHARED_MEMORY=true TCC_MEASUREMENTS_TIME_UNIT=us TCC_MEASUREMENTS_DUMP_FILE=dump.txt TCC_MEASUREMENTS_BUFFERS=Input:10:10,Compute:10:100 ./myapp

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.