Intel® oneAPI Deep Neural Network Developer Guide and Reference

Download PDF
ID 768875
Date 6/30/2025
Public
Document Table of Contents
Document Table of Contents x
oneAPI Deep Neural Network Library (oneDNN) Developer Guide and Reference
oneAPI Deep Neural Network Library (oneDNN) Developer Guide and Reference x
Building and Linking Programming Model Supported Primitives Graph Extension Examples Performance Profiling and Inspection Advanced Topics Ukernels oneDNN API genindex
Building and Linking x
Build from Source Build Options Linking to the Library
Programming Model x
Basic Concepts Getting started Memory Format Propagation Inference and Training Aspects Primitive Attributes Data Types Reorder between CPU and GPU engines API Interoperability with DPC++ and OpenCL
Inference and Training Aspects x
Inference Int8 Inference Bfloat16 Training
Primitive Attributes x
Primitive Attributes: floating-point math mode Primitive Attributes: accumulation mode Primitive Attributes: rounding mode Primitive Attributes: deterministic Primitive Attributes: Quantization Primitive Attributes: Post-ops Primitive Attributes: Scratchpad
Interoperability with DPC++ and OpenCL x
OpenCL Interoperability DPC++ Interoperability
Supported Primitives x
Convolution Inner Product Matrix Multiplication RNN Batch Normalization Binary Concat Eltwise Group Normalization Layer Normalization Local Response Normalization (LRN) Pooling PReLU Resampling Shuffle Softmax Sum Reorder Reduction
Graph Extension x
Programming Model Supported Operations Fusion Patterns Graph Dump Constant Tensor Cache
Programming Model x
Basic Concepts Low Precision
Supported Operations x
Abs AbsBackward Add AvgPool AvgPoolBackward BatchNormForwardTraining BatchNormInference General Operation attributes Execution arguments Supported data types BatchNormTrainingBackward BiasAdd BiasAddBackward Clamp ClampBackward Concat Convolution ConvolutionBackwardData ConvolutionBackwardWeights ConvTranspose ConvTransposeBackwardData ConvTransposeBackwardWeights Dequantize Divide DynamicDequantize DynamicQuantize Elu EluBackward End Exp GroupNorm GELU GELUBackward HardSigmoid HardSigmoidBackward HardSwish HardSwishBackward Interpolate InterpolateBackward LayerNorm LayerNormBackward LeakyReLU Log LogSoftmax LogSoftmaxBackward MatMul Maximum MaxPool MaxPoolBackward Minimum Mish MishBackward Multiply Pow PReLU PReLUBackward Quantize Reciprocal ReduceL1 ReduceL2 ReduceMax ReduceMean ReduceMin ReduceProd ReduceSum ReLU ReLUBackward Reorder Round Select Sigmoid SigmoidBackward SoftMax SoftMaxBackward SoftPlus SoftPlusBackward Sqrt SqrtBackward Square SquaredDifference StaticReshape StaticTranspose Subtract Tanh TanhBackward TypeCast Wildcard
Fusion Patterns x
Gated Multi-Layer Perceptron (Gated-MLP) Grouped Query Attention (GQA) SDPA with Compressed Key and Value Scaled Dot-Product Attention (SDPA)
Performance Profiling and Inspection x
Verbose Mode Configuring oneDNN for Benchmarking Benchmarking Performance Profiling oneDNN Performance Inspecting JIT Code Performance Profiling Example CPU Dispatcher Control CPU ISA Hints Verbose Message Catalogue
Advanced Topics x
Transitioning from Intel MKL-DNN to oneDNN Understanding Memory Formats Nuances of int8 Computations Primitive Cache Persistent Cache Using oneDNN with Threadpool-Based Threading Experimental features
Ukernels x
Basic Concepts Batch-Reduce General Matrix Multiplication Data transformation BRGeMM ukernel example
oneDNN API x
Primitives Primitive Cache BLAS functions Common API Graph API Runtime interoperability API Ukernels namespace dnnl namespace oneapi
Primitives x
Common Attributes Reorder Concat Sum Binary Convolution Deconvolution Shuffle Eltwise Softmax Pooling PReLU LRN Batch Normalization Group Normalization Layer Normalization Inner Product RNN Matrix Multiplication Resampling Reduction
Common x
enum dnnl_alg_kind_t enum dnnl_normalization_flags_t enum dnnl_primitive_kind_t enum dnnl_prop_kind_t enum dnnl_query_t enum dnnl::normalization_flags enum dnnl::query struct dnnl_exec_arg_t struct dnnl_primitive struct dnnl_primitive_desc struct dnnl::primitive struct dnnl::primitive_desc struct dnnl::primitive_desc_base
struct dnnl::primitive x
enum dnnl::primitive::kind
Attributes x
enum dnnl::algorithm enum dnnl_rounding_mode_t enum dnnl_scratchpad_mode_t enum dnnl::prop_kind enum dnnl::rounding_mode enum dnnl::scratchpad_mode struct dnnl_post_ops struct dnnl_primitive_attr struct dnnl::post_ops struct dnnl::primitive_attr
Reorder x
struct dnnl::reorder
struct dnnl::reorder x
struct dnnl::reorder::primitive_desc
Concat x
struct dnnl::concat
struct dnnl::concat x
struct dnnl::concat::primitive_desc
Sum x
struct dnnl::sum
struct dnnl::sum x
struct dnnl::sum::primitive_desc
Binary x
struct dnnl::binary
struct dnnl::binary x
struct dnnl::binary::primitive_desc
Convolution x
struct dnnl::convolution_backward_data struct dnnl::convolution_backward_weights struct dnnl::convolution_forward
struct dnnl::convolution_backward_data x
struct dnnl::convolution_backward_data::primitive_desc
struct dnnl::convolution_backward_weights x
struct dnnl::convolution_backward_weights::primitive_desc
struct dnnl::convolution_forward x
struct dnnl::convolution_forward::primitive_desc
Deconvolution x
struct dnnl::deconvolution_backward_data struct dnnl::deconvolution_backward_weights struct dnnl::deconvolution_forward
struct dnnl::deconvolution_backward_data x
struct dnnl::deconvolution_backward_data::primitive_desc
struct dnnl::deconvolution_backward_weights x
struct dnnl::deconvolution_backward_weights::primitive_desc
struct dnnl::deconvolution_forward x
struct dnnl::deconvolution_forward::primitive_desc
Shuffle x
struct dnnl::shuffle_backward struct dnnl::shuffle_forward
struct dnnl::shuffle_backward x
struct dnnl::shuffle_backward::primitive_desc
struct dnnl::shuffle_forward x
struct dnnl::shuffle_forward::primitive_desc
Eltwise x
struct dnnl::eltwise_backward struct dnnl::eltwise_forward
struct dnnl::eltwise_backward x
struct dnnl::eltwise_backward::primitive_desc
struct dnnl::eltwise_forward x
struct dnnl::eltwise_forward::primitive_desc
Softmax x
struct dnnl::softmax_backward struct dnnl::softmax_forward
struct dnnl::softmax_backward x
struct dnnl::softmax_backward::primitive_desc
struct dnnl::softmax_forward x
struct dnnl::softmax_forward::primitive_desc
Pooling x
struct dnnl::pooling_backward struct dnnl::pooling_forward
struct dnnl::pooling_backward x
struct dnnl::pooling_backward::primitive_desc
struct dnnl::pooling_forward x
struct dnnl::pooling_forward::primitive_desc
PReLU x
struct dnnl::prelu_backward struct dnnl::prelu_forward
struct dnnl::prelu_backward x
struct dnnl::prelu_backward::primitive_desc
struct dnnl::prelu_forward x
struct dnnl::prelu_forward::primitive_desc
LRN x
struct dnnl::lrn_backward struct dnnl::lrn_forward
struct dnnl::lrn_backward x
struct dnnl::lrn_backward::primitive_desc
struct dnnl::lrn_forward x
struct dnnl::lrn_forward::primitive_desc
Batch Normalization x
struct dnnl::batch_normalization_backward struct dnnl::batch_normalization_forward
struct dnnl::batch_normalization_backward x
struct dnnl::batch_normalization_backward::primitive_desc
struct dnnl::batch_normalization_forward x
struct dnnl::batch_normalization_forward::primitive_desc
Group Normalization x
struct dnnl::group_normalization_backward struct dnnl::group_normalization_forward
struct dnnl::group_normalization_backward x
struct dnnl::group_normalization_backward::primitive_desc
struct dnnl::group_normalization_forward x
struct dnnl::group_normalization_forward::primitive_desc
Layer Normalization x
struct dnnl::layer_normalization_backward struct dnnl::layer_normalization_forward
struct dnnl::layer_normalization_backward x
struct dnnl::layer_normalization_backward::primitive_desc
struct dnnl::layer_normalization_forward x
struct dnnl::layer_normalization_forward::primitive_desc
Inner Product x
struct dnnl::inner_product_backward_data struct dnnl::inner_product_backward_weights struct dnnl::inner_product_forward
struct dnnl::inner_product_backward_data x
struct dnnl::inner_product_backward_data::primitive_desc
struct dnnl::inner_product_backward_weights x
struct dnnl::inner_product_backward_weights::primitive_desc
struct dnnl::inner_product_forward x
struct dnnl::inner_product_forward::primitive_desc
RNN x
enum dnnl_rnn_direction_t enum dnnl_rnn_flags_t enum dnnl::rnn_direction enum dnnl::rnn_flags struct dnnl::augru_backward struct dnnl::augru_forward struct dnnl::gru_backward struct dnnl::gru_forward struct dnnl::lbr_augru_backward struct dnnl::lbr_augru_forward struct dnnl::lbr_gru_backward struct dnnl::lbr_gru_forward struct dnnl::lstm_backward struct dnnl::lstm_forward struct dnnl::rnn_primitive_desc_base struct dnnl::vanilla_rnn_backward struct dnnl::vanilla_rnn_forward
struct dnnl::augru_backward x
struct dnnl::augru_backward::primitive_desc
struct dnnl::augru_forward x
struct dnnl::augru_forward::primitive_desc
struct dnnl::gru_backward x
struct dnnl::gru_backward::primitive_desc
struct dnnl::gru_forward x
struct dnnl::gru_forward::primitive_desc
struct dnnl::lbr_augru_backward x
struct dnnl::lbr_augru_backward::primitive_desc
struct dnnl::lbr_augru_forward x
struct dnnl::lbr_augru_forward::primitive_desc
struct dnnl::lbr_gru_backward x
struct dnnl::lbr_gru_backward::primitive_desc
struct dnnl::lbr_gru_forward x
struct dnnl::lbr_gru_forward::primitive_desc
struct dnnl::lstm_backward x
struct dnnl::lstm_backward::primitive_desc
struct dnnl::lstm_forward x
struct dnnl::lstm_forward::primitive_desc
struct dnnl::vanilla_rnn_backward x
struct dnnl::vanilla_rnn_backward::primitive_desc
struct dnnl::vanilla_rnn_forward x
struct dnnl::vanilla_rnn_forward::primitive_desc
Matrix Multiplication x
struct dnnl::matmul
struct dnnl::matmul x
struct dnnl::matmul::primitive_desc
Resampling x
struct dnnl::resampling_backward struct dnnl::resampling_forward
struct dnnl::resampling_backward x
struct dnnl::resampling_backward::primitive_desc
struct dnnl::resampling_forward x
struct dnnl::resampling_forward::primitive_desc
Reduction x
struct dnnl::reduction
struct dnnl::reduction x
struct dnnl::reduction::primitive_desc
Common API x
Engine Stream Floating-point Math Mode Service Utilities Accumulation Mode Data types Memory
Engine x
enum dnnl_engine_kind_t struct dnnl_engine struct dnnl::engine
struct dnnl::engine x
enum dnnl::engine::kind
Stream x
enum dnnl_stream_flags_t struct dnnl_stream struct dnnl::stream
struct dnnl::stream x
enum dnnl::stream::flags
Floating-point Math Mode x
enum dnnl_fpmath_mode_t enum dnnl::fpmath_mode
Service x
enum dnnl::cpu_isa enum dnnl::cpu_isa_hints enum dnnl_cpu_isa_hints_t enum dnnl_cpu_isa_t enum dnnl::status struct dnnl_version_t
Utilities x
enum dnnl_status_t struct dnnl::error template struct dnnl::handle template struct dnnl::handle_traits
Accumulation Mode x
enum dnnl::accumulation_mode enum dnnl_accumulation_mode_t
Data types x
enum dnnl_data_type_t
Memory x
enum dnnl_format_kind_t enum dnnl_format_tag_t enum dnnl_sparse_encoding_t struct dnnl_memory struct dnnl_memory_desc struct dnnl::memory
struct dnnl::memory x
enum dnnl::memory::data_type enum dnnl::memory::format_kind enum dnnl::memory::format_tag enum dnnl::memory::sparse_encoding struct dnnl::memory::desc
Graph API x
Allocator Engine Logical Tensor Tensor Op Partition Compiled Partition Graph Compiled Partition Cache Constant Tensor Cache Runtime interoperability API namespace dnnl::graph
Allocator x
class dnnl::graph::allocator
Logical Tensor x
enum dnnl_graph_layout_type_t enum dnnl_graph_tensor_property_t struct dnnl_graph_logical_tensor_t class dnnl::graph::logical_tensor
class dnnl::graph::logical_tensor x
enum dnnl::graph::logical_tensor::data_type enum dnnl::graph::logical_tensor::layout_type enum dnnl::graph::logical_tensor::property_type
Tensor x
class dnnl::graph::tensor
Op x
enum dnnl_graph_op_attr_t enum dnnl_graph_op_kind_t class dnnl::graph::op
class dnnl::graph::op x
enum dnnl::graph::op::attr enum dnnl::graph::op::kind
Partition x
enum dnnl_graph_partition_policy_t class dnnl::graph::partition
class dnnl::graph::partition x
enum dnnl::graph::partition::policy
Compiled Partition x
struct dnnl_graph_inplace_pair_t class dnnl::graph::compiled_partition
Graph x
class dnnl::graph::graph
Runtime interoperability API x
OpenCL interoperability API SYCL interoperability API
OpenCL interoperability API x
namespace dnnl::graph::ocl_interop
SYCL interoperability API x
namespace dnnl::graph::sycl_interop
Runtime interoperability API x
OpenCL interoperability API SYCL interoperability API Threadpool interoperability API
OpenCL interoperability API x
namespace dnnl::ocl_interop enum dnnl_ocl_interop_memory_kind_t
namespace dnnl::ocl_interop x
enum dnnl::ocl_interop::memory_kind
SYCL interoperability API x
namespace dnnl::sycl_interop enum dnnl_sycl_interop_memory_kind_t
namespace dnnl::sycl_interop x
enum dnnl::sycl_interop::memory_kind
Threadpool interoperability API x
namespace dnnl::threadpool_interop
namespace dnnl::threadpool_interop x
struct dnnl::threadpool_interop::threadpool_iface
Ukernels x
BRGeMM ukernel namespace dnnl::ukernel enum dnnl_pack_type_t struct dnnl_ukernel_attr_params
BRGeMM ukernel x
struct dnnl::ukernel::brgemm struct dnnl_brgemm struct dnnl_transform
oneAPI Deep Neural Network Library (oneDNN) Developer Guide and Reference

BatchNormInference

Level Two Title