Softmax
General
The softmax primitive performs forward or backward softmax or logsoftmax operation along a particular axis on data with arbitrary dimensions. All other axes are treated as independent (batch).
Forward
In general form, the operation is defined by the following formulas (the variable names follow the standard Naming Conventions).
Softmax:

Logsoftmax:

Above
is the axis over which the operation is computed on,
is the outermost index (to the left of the axis),
is the innermost index (to the right of the axis), and
is used to produce numerically stable results and defined as:
Difference Between Forward Training and Forward Inference
There is no difference between the dnnl_forward_training and dnnl_forward_inference propagation kinds.
Backward
The backward propagation computes
, based on
and
.



Execution Arguments
When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table.
Primitive input/output | Execution argument index |
---|---|
![]() | DNNL_ARG_SRC |
![]() | DNNL_ARG_DST |
![]() | DNNL_ARG_DIFF_SRC |
![]() | DNNL_ARG_DIFF_DST |
Implementation Details
General Notes
- Both forward and backward propagation support in-place operations, meaning thatsrccan be used as input and output for forward propagation, anddiff_dstcan be used as input and output for backward propagation. In case of in-place operation, the original data will be overwritten. This support is limited to cases when data types ofsrc/dstordiff_src/diff_dstare identical.
Post-ops and Attributes
Attributes enable you to modify the behavior of the softmax primitive. The following attributes are supported by the softmax primitive:
Propagation | Type | Operation | Description | Restrictions |
---|---|---|---|---|
forward | attribute | Output scale | Scales the result of softmax by given scale factor | int8 softmax only, zero mask only |
Data Type Support
The softmax primitive supports the following combinations of data types:
Propagation | Source | Destination |
---|---|---|
forward / backward | f32, bf16 | f32, bf16 |
forward | f16 | f16 |
forward | f32, bf16, u8, s8 | u8, s8 |
forward | u8, s8 | f32, bf16 |
Data Representation
Source, Destination, and Their Gradients
The softmax primitive works with arbitrary data tensors. There is no special meaning associated with any logical dimensions. However, the softmax axis is typically referred to as channels (hence in formulas we use
).

Implementation Limitations
- No primitive specific limitations. Refer to Data Types for limitations related to data types support.
Performance Tips
- Use in-place operations whenever possible.
- Currently the softmax primitive is optimized for the cases where the dimension of the softmax axis is physically dense. For instance:
Example
This C++ API example demonstrates how to create and execute a Softmax primitive in forward training propagation mode.
Key optimizations included in this example:
- In-place primitive execution;
- Softmax along axis 1 (C) for 2D tensors.