Developer Reference for Intel® oneAPI Math Kernel Library for C
Inspector-Executor Sparse BLAS Verbose Mode
Inspector-Executor Sparse BLAS routines can report diagnostic information about the calls they execute to the Intel® oneAPI Math Kernel Library (oneMKL) verbose output. This domain’s verbose output format departs from that of other oneMKL domains: each traced call emits a BEGIN line naming the routine and its arguments, followed by a separate END line reporting the timing and the execution environment of the call.
Enabling Verbose Mode
Inspector-Executor Sparse BLAS verbose output is controlled entirely by the oneMKL verbose switch, which is shared with other oneMKL domains:
the mkl_verbose support function, or
the MKL_VERBOSE environment variable.
Enabling verbose output with any of these options, will turn on logging for all domains.
By default, verbose output is written to stdout. Use the mkl_verbose_output_file support function or the MKL_VERBOSE_OUTPUT_FILE environment variable to redirect it to a file.
Verbose Call Trace
Each traced call prints two lines rather than the single call description line used by other Intel® oneAPI Math Kernel Library (oneMKL) domains:
MKL_VERBOSE BEGIN <name>(<parameter tokens>)
MKL_VERBOSE END <name>() <time> CNR:<branch> Dyn:<0|1> FastMM:<0|1> TID:<n> NThr:<n><name> is the routine name. When it is applicable, the name of the routine contains the letter corresponding to the requested data type, for example mkl_sparse_d_create_csr for double precision.
The BEGIN line lists the routine’s parameters as <parameter tokens> , a sequence of comma-separated, named tokens giving each input value. See Parameter Tokens for the list of available tokens and their meaning.
The END line reports the elapsed time of the call followed by the same environment fields printed in the call description line of every other domain: CNR , Dyn , FastMM , TID , and NThr . See Call Description Line (Linux) or Call Description Line (Windows) for the meaning of these fields.
A BEGIN / END pair, together with anything printed between them, may be referred to as a frame or verbose frame in the subsequent.
The BEGIN / END pair exists to leave room for future output: although none exists today, a later release could print extra lines between a call’s BEGIN and END , corresponding to different levels of detail in Sparse BLAS verbose output.
A call’s BEGIN / END pair is never nested inside, or interleaved with, another call’s pair: verbose frames are printed one after another, in the order the calls complete.
Verbose mode is not MPI-aware, however: across ranks, lines can still interleave if they are written to the same destination, whether stdout or a file set with mkl_verbose_output_file . As with every other Intel® oneAPI Math Kernel Library (oneMKL) domain, we recommend directing each rank’s verbose output to its own file, for example by including the rank number in the output filename.
Parameter Tokens
The list of tokens below is used in the verbose output of all Inspector-Executor Sparse BLAS routines. A given routine prints only the tokens that apply to its own parameters, in the order those parameters appear in its signature.
A matrix handle parameter is printed as a bracketed dump built from these fields:
Field |
Meaning |
|---|---|
fmt |
Storage format: coo , csr , csc , or bsr . |
dims |
Dimensions of a coo , csr , or csc matrix, printed as rows x cols . |
nnz |
Number of nonzeros of a coo , csr , or csc matrix, or n/a if not yet known (for example, for an output handle before the call has produced a result). |
blk_dims |
Block-dimensions of a bsr matrix, printed as rows x cols of blocks. |
blk_shape |
Block shape and layout of a bsr matrix, printed as row_block_size x col_block_size followed by RM (row-major) or CM (column-major). |
blk_nnz |
Number of nonzero blocks of a bsr matrix. |
indexing |
0-based or 1-based . |
prec |
Data type (precision) of the matrix values: s , d , c , or z . |
mat_id |
A process-unique ID assigned to the matrix handle when it is created; useful to easily distinguish between multiple matrices potentially in use at the same time in the application. |
Other tokens that appear directly to describe the routine’s parameters:
Token |
Meaning |
|---|---|
int |
Interface width used for MKL_INT : i4 (32-bit) or i8 (64-bit). Printed by routines whose parameters include MKL_INT values; omitted by routines whose own parameters do not. |
op |
Requested operation: N , T , or C for non-transpose, transpose, or conjugate-transpose. |
descr |
Matrix descriptor, printed as [type/fill/diag] :
|
layout |
RM or CM for row-major or column-major, used to specify the layout of dense structures. |
req |
Two-stage request stage: FULL , NNZ_COUNT , FINALIZE , FULL_NO_VAL , or FINALIZE_NO_VAL . See Two-Stage Algorithms . |
sor |
SOR sweep direction: FWD (forward), BWD (backward), or SYM (symmetric). |
calls |
The expected-calls count passed to a set_*_hint routine. |
alpha , beta |
Scalar coefficients, printed by value; a complex scalar is printed as (real,imag) . |
Array and pointer parameters, for example x , y , values , col_indx , rows_start |
Printed as an address, 0x followed by hexadecimal digits, to show the alignment of the data rather than its contents. |
p_<name> |
The address of an output handle pointer, for example p_A . Immediately followed by the <name>:[...] dump of the matrix that handle points to (or will point to once the call succeeds). |
A few conventions apply throughout a frame:
Any enum value that the frame does not recognize is printed as ? .
A NULL matrix handle is printed as [null] in place of the bracketed address dump.
A create routine that produced no handle reports mat_id:-1 .
Verbose Output Examples
The following sequence represents potential output from the creation of a 5x5 CSR matrix handle, setting a matrix-vector multiplication hint on it, optimizing it, performing a matrix-vector multiplication operation, and finally destroying the handle, with verbose mode enabled throughout. Addresses, number of threads, matrix ids, and timings are illustrative.
MKL_VERBOSE BEGIN mkl_sparse_d_create_csr(int:i4,mat_id:1,rows:5,cols:5,indexing:0-based,rows_start:0x7f1a2c000010,rows_end:0x7f1a2c000030,col_indx:0x7f1a2c000050,values:0x7f1a2c000070)
MKL_VERBOSE END mkl_sparse_d_create_csr() 1.20us CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:8
MKL_VERBOSE BEGIN mkl_sparse_set_mv_hint(int:i4,A:[fmt:csr,dims:5x5,nnz:13,indexing:0-based,prec:d,mat_id:1],op:N,descr:[GEN/F/N],calls:1)
MKL_VERBOSE END mkl_sparse_set_mv_hint() 640ns CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:8
MKL_VERBOSE BEGIN mkl_sparse_optimize(A:[fmt:csr,dims:5x5,nnz:13,indexing:0-based,prec:d,mat_id:1])
MKL_VERBOSE END mkl_sparse_optimize() 3.45us CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:8
MKL_VERBOSE BEGIN mkl_sparse_d_mv(op:N,alpha:1,A:[fmt:csr,dims:5x5,nnz:13,indexing:0-based,prec:d,mat_id:1],descr:[GEN/F/N],beta:0,x:0x7f1a2c0000a0,y:0x7f1a2c0000c0)
MKL_VERBOSE END mkl_sparse_d_mv() 890ns CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:8
MKL_VERBOSE BEGIN mkl_sparse_destroy(A:[fmt:csr,dims:5x5,nnz:13,indexing:0-based,prec:d,mat_id:1])
MKL_VERBOSE END mkl_sparse_destroy() 210ns CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:8Performance Considerations
Enabling verbose mode adds a fixed cost to every traced call, which is largely independent of the size of the matrix or the operation being performed. Because this cost is paid on every call, it is recommended to disable verbose mode in production runs, and to enable it only while diagnosing or profiling an application.