Developer Guide and Reference

  • 2022.1
  • 04/11/2022
  • Public Content
Contents

Batch Normalization

Overview

A primitive to perform batch normalization. More…
// structs struct dnnl::batch_normalization_backward; struct dnnl::batch_normalization_forward; struct dnnl_batch_normalization_desc_t; // global functions dnnl_status_t DNNL_API dnnl_batch_normalization_forward_desc_init( dnnl_batch_normalization_desc_t* bnrm_desc, dnnl_prop_kind_t prop_kind, const dnnl_memory_desc_t* data_desc, float epsilon, unsigned flags ); dnnl_status_t DNNL_API dnnl_batch_normalization_backward_desc_init( dnnl_batch_normalization_desc_t* bnrm_desc, dnnl_prop_kind_t prop_kind, const dnnl_memory_desc_t* diff_data_desc, const dnnl_memory_desc_t* data_desc, float epsilon, unsigned flags );

Detailed Documentation

A primitive to perform batch normalization.
Both forward and backward propagation primitives support in-place operation; that is, src and dst can refer to the same memory for forward propagation, and diff_dst and diff_src can refer to the same memory for backward propagation.
The batch normalization primitives computations can be controlled by specifying different dnnl::normalization_flags values. For example, batch normalization forward propagation can be configured to either compute the mean and variance or take them as arguments. It can either perform scaling and shifting using gamma and beta parameters or not. Optionally, it can also perform a fused ReLU, which in case of training would also require a workspace.
See also:
Batch Normalization in developer guide
Global Functions
dnnl_status_t DNNL_API dnnl_batch_normalization_forward_desc_init( dnnl_batch_normalization_desc_t* bnrm_desc, dnnl_prop_kind_t prop_kind, const dnnl_memory_desc_t* data_desc, float epsilon, unsigned flags )
Initializes a descriptor for a batch normalization forward propagation primitive.
In-place operation is supported: the dst can refer to the same memory as the src.
Parameters:
bnrm_desc
Output descriptor for batch normalization primitive.
prop_kind
Propagation kind. Possible values are dnnl_forward_training and dnnl_forward_inference.
data_desc
Source and destination memory descriptor.
epsilon
Batch normalization epsilon parameter.
flags
Batch normalization flags (dnnl_normalization_flags_t).
Returns:
dnnl_success on success and a status describing the error otherwise.
dnnl_status_t DNNL_API dnnl_batch_normalization_backward_desc_init( dnnl_batch_normalization_desc_t* bnrm_desc, dnnl_prop_kind_t prop_kind, const dnnl_memory_desc_t* diff_data_desc, const dnnl_memory_desc_t* data_desc, float epsilon, unsigned flags )
Initializes a descriptor for a batch normalization backward propagation primitive.
In-place operation is supported: the diff_dst can refer to the same memory as the diff_src.
Parameters:
bnrm_desc
Output descriptor for batch normalization primitive.
prop_kind
Propagation kind. Possible values are dnnl_backward_data and dnnl_backward (diffs for all parameters are computed in this case).
diff_data_desc
Diff source and diff destination memory descriptor.
data_desc
Source memory descriptor.
epsilon
Batch normalization epsilon parameter.
flags
Batch normalization flags (dnnl_normalization_flags_t).
Returns:
dnnl_success on success and a status describing the error otherwise.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.