Recording Communication Events
These are API calls that allow logging of message send and receive and MPI-style collective operations. Because they are modeled after MPI operations, they use the same kind of communicator to define the context for the operation.
enum _VT_CommIDs
Logging send/receive events evaluates the rank local within the active communicator, and matches events only if they take place in the same communicator (in other words, it is the same behavior as in MPI).
Defining new communicators is not supported, but the predefined ones can be used.
Enumerator
| Description
|
|---|---|
VT_COMM_INVALID | Invalid ID, do not pass to Intel® Trace Collector
|
VT_COMM_WORLD | Global ranks are the same as local ones
|
VT_COMM_SELF | Communicator that only contains the active process
|
VT_log_sendmsg
int VT_log_sendmsg(int other_rank, int count, int tag, int commid, int sclhandle)
Description
Logs sending of a message.
Fortran
VTLOGSENDMSG(other_rank, count, tag, commid, sclhandle, ierr)
Parameters
my_rank | rank of the sending process
|
other_rank | rank of the target process
|
count | number of bytes sent
|
tag | tag of the message
|
commid | numeric ID for the communicator ( VT_COMM_WORLD ,
VT_COMM_SELF , or see
VT_commdef() )
|
sclhandle | handle as defined by
VT_scldef , or
VT_NOSCL |
Return values
Returns error code
VT_log_recvmsg
int VT_log_recvmsg(int other_rank, int count, int tag, int commid, int sclhandle)
Description
Logs receiving of a message.
Fortran
VTLOGRECVMSG(other_rank, count, tag, commid, sclhandle, ierr)
Parameters
my_rank | rank of the receiving process
|
other_rank | rank of the source process
|
count | number of bytes sent
|
tag | tag of the message
|
commid | numeric ID for the communicator ( VT_COMM_WORLD ,
VT_COMM_SELF , or see
VT_commdef() )
|
sclhandle | handle as defined by
VT_scldef , or
VT_NOSCL |
Return values
Returns error code
The next three calls require a little extra care, because they generate events that not only have a time stamp, but also a duration. This means that you need to take a time stamp first, then do the operation and finally log the event.
VT_log_msgevent
int VT_log_msgevent(int sender, int receiver, int count, int tag, int commid, double sendts, int sendscl, int recvscl)
Description
Logs sending and receiving of a message.
Fortran
VTLOGMSGEVENT(sender, receiver, count, tag, commid, sendts, sendscl, recvscl, ierr)
Parameters
sender | rank of the sending process
|
receiver | rank of the target process
|
count | number of bytes sent
|
tag | tag of the message
|
commid | numeric ID for the communicator ( VT_COMM_WORLD ,
VT_COMM_SELF , or see
VT_commdef() )
|
sendts | time stamp obtained with
VT_timestamp() |
sendscl | handle as defined by
VT_scldef() for the source code location where the message was sent, or
VT_NOSCL |
recvscl | the same for the receive location
|
Return values
Returns error code
VT_log_op
int VT_log_op(int opid, int commid, int root, int bsend, int brecv, double startts, int sclhandle)
Description
Logs the duration and amount of transferred data of an operation for one process.
Fortran
VTLOGOP(opid, commid, root, bsend, brecv, startts, sclhandle, ierr)
Parameters
opid | id of the operation; must be one of the predefined constants in
enum _VT_OpTypes |
commid | numeric ID for the communicator; see
VT_log_sendmsg() for valid numbers
|
root | rank of the root process in the communicator (ignored for operations without root, must still be valid, though)
|
bsend | bytes sent by process (ignored for operations that send no data)
|
brecv | bytes received by process (ignored for operations that receive no data)
|
startts | the start time of the operation (as returned by
VT_timestamp() )
|
sclhandle | handle as defined by
VT_scldef , or
VT_NOSCL |
Return values
Returns error code
VT_log_opevent
int VT_log_opevent(int opid, int commid, int root, int numprocs, int _ bsend, int _ brecv, double _ startts, int sclhandle)
Description
Logs the duration and amount of transferred data of an operation for all involved processes at once.
Intel® Trace Collector knows which processes send and receive data in each operation. Unused byte counts are ignored when writing the trace, so they can be left uninitialized, but NULL is not allowed as array address even if no entry is used at all.
Fortran
VTLOGOPEVENT(opid, commid, root, numprocs, bsend, brecv, startts, sclhandle, ierr)
Parameters
opid | id of the operation; must be one of the predefined constants in
enum _VT_OpTypes |
commid | numeric ID for the communicator; see
VT_log_sendmsg() for valid numbers
|
root | rank of the root process in the communicator (ignored for operations without root, must still be valid, though)
|
numprocs | the number of processes in the communicator
|
bsend | bytes sent by process
|
brecv | bytes received by process
|
startts | the start time of the operation (as returned by
VT_timestamp() )
|
sclhandle | handle as defined by
VT_scldef , or
VT_NOSCL |
Return values
Returns error code
enum _VT_OpTypes
These are operation IDs that can be passed to
VT_log_op()
.
Their representation in the trace file matches that of the equivalent MPI operation.
User-defined operations are not supported.
Enumerator
| Description
|
|---|---|
| Undefined operation, should not be passed to Intel® Trace Collector
|
| Number of predefined operations
|
| MPI operation representations
|
Having a duration also may introduce the problem of having overlapping operations, which has to be taken care of with the following two calls.
VT_begin_unordered
int VT_begin_unordered(void)
Description
Starts a period with out-of-order events.
Most API functions log events with just one time stamp which is taken when the event is logged. That guarantees strict chronological order of the events.
VT_log_msgevent()
and
VT_log_opevent()
are logged when the event has finished with a start time taken earlier with
VT_timestamp()
. This can break the chronological order, for example, like in the following two examples:
t1: VT_timestamp() "start message" t2: VT_end() "leave function" t3: VT_log_msgevent( t1 ) "finish message"
t1: VT_timestamp() "start first message" t2: VT_timestamp() "start second message" t3: VT_log_msgevent( t1 ) "finish first message" t4: VT_log_msgevent( t2 ) "finish second message"
In other words, it is okay to just log a complex event if and only if no other event is logged between its start and end in this thread. "logged" in this context includes other complex events that are logged later, but with a start time between the other events start and end time.
In all other cases you have to alert Intel® Trace Collector of the fact that out-of-order events will follow by calling
VT_begin_unordered()
before and
VT_end_unordered()
after these events. When writing the events into the trace file Intel® Trace Collector increases a counter per thread when it sees a
VT_begin_unordered()
and decrease it at a
VT_end_unordered()
. Events are remembered and sorted until the counter reaches zero, or till the end of the data.
This means that:
- unordered periods can be nested
- it is not necessary to close each unordered period at the end of the trace
- but not closing them properly in the middle of a trace will force Intel® Trace Collector to use a lot more memory when writing the trace (proportional to the number of events till the end of the trace).
Fortran
VTBEGINUNORDERED(ierr)
VT_end_unordered
int VT_end_unordered (void)
Description
Close a period with out-of-order events that was started with
VT_begin_unordered()
.
Fortran
VTENDNORDERED(ierr)