Application Performance Snapshot User Guide for Linux* OS

ID 772048
Date 3/31/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Common Analyses

This section describes report types that are available for all applications, whether they use MPI or not.

Counters Report

Key:-o [--counters]

This report displays the minimum, maximum, and average values of all collected MPI and hardware metrics. This report also shows total values for metrics, where applicable. This report also shows the node or rank where the minimum or maximum value was observed next to the value.

Example:

| Counters, Memory and Disk usage statistics for all Ranks
|---------------------------------------------------------
Computation:           40.11%
MPI        :           59.89%

| Virtual Memory Usage per Rank:
          Total:    134246164.66 MB (all ranks)
            MIN:     33560779.87 MB (rank 3)
            MAX:     33562304.46 MB (rank 0)
            AVG:     33561541.16 MB

| Resident Memory Usage per Rank:
          Total:         1450.39 MB (all ranks)
            MIN:          129.72 MB (rank 1)
            MAX:          593.68 MB (rank 0)
            AVG:          362.60 MB

| MPI Time per Rank:
          Total:           13.48 sec (all ranks)
            MIN:            1.12 sec (rank 2)
            MAX:            6.01 sec (rank 1)
            AVG:            3.37 sec

[...]

| SP GFLOPS
          Total:            0.00
            MIN:            0.00 (node s011-n005)
            MAX:            0.00 (node s011-n005)
            AVG:            0.00
            
[...]

Node Topology Report

Key:--node-topology

This report shows the association between ranks, nodes, and PCIe devices in the collected data. When the rank filter --rank is applied, the report displays node information for nodes where the specified rank was executed.

Example:

| Node topology
|---------------------------------------------------------
Node s011-n004:
  ranks: 0-1
  devices:
    Bridge: Intel Corporation Device 0x2020 0000:00:00.0
    Display controller: Intel Corporation Device 0x4905 0000:1c:00.0
    Display controller: Intel Corporation Device 0x4905 0000:6a:00.0
    Mass storage controller: Intel Corporation Device 0xfaf0 0000:b3:00.0
Node s011-n005:
  ranks: 2-3
  devices:
    Bridge: Intel Corporation Device 0x2020 0000:00:00.0
    Display controller: Intel Corporation Device 0x4905 0000:1c:00.0
    Display controller: Intel Corporation Device 0x4905 0000:6a:00.0
    Mass storage controller: Intel Corporation Device 0xfaf0 0000:b3:00.0

Metrics Report

Key:--metrics arg

This report shows a table with performance metrics for each rank, node, and PCIe device.

You can specify the exact metrics to be displayed in the table.

Use --metrics=? to get the list of metrics available in the collected data.

Use --metrics=all to display all available metrics.

Use --metrics="Metric 1, Metric 2, Metric 3" to display a specific set of metrics.

Example:

| Metric Table
|---------------------------------------------------------
Rank    Node Name       Device Type     Device Name     Metric Name     Metric Value
0       s011-n004       N/A     N/A     MPI Time, usec  1.37737e+06
N/A     s011-n004       N/A     N/A     Elapsed Time, sec       6.72061
N/A     s011-n004       GPU     Display controller: Intel Corporation Device 0x4905 0000:1c:00.0        Inbound PCIe Write, MB/s        103.57
N/A     s011-n004       Other   Mass storage controller: Intel Corporation Device 0xfaf0 0000:b3:00.0   Inbound PCIe Write, MB/s        29.9996
N/A     s011-n004       Other   Bridge: Intel Corporation Device 0x2020 0000:00:00.0    Inbound PCIe Write, MB/s       3.55524
N/A     s011-n004       GPU     Display controller: Intel Corporation Device 0x4905 0000:6a:00.0        Inbound PCIe Write, MB/s        0