Intel® oneAPI Deep Neural Network Developer Guide and Reference
A newer version of this document is available. Customers should click here to go to the newest version.
Visible to Intel only — GUID: GUID-00FD0F1F-4FA5-4930-B33B-83A0930A1071
Visible to Intel only — GUID: GUID-00FD0F1F-4FA5-4930-B33B-83A0930A1071
Local Response Normalization (LRN)
General
The LRN primitive performs a forward or backward local response normalization.
Forward
The LRN operation is defined by the following formulas (the variable names follow the standard Naming Conventions):
LRN across channels :
LRN within channel :
where is the local_size. Formulas are provided for 2D spatial data case.
Backward
The backward propagation computes 
, based on and 
.
Execution Arguments
When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table.
Primitive input/output |
Execution argument index |
|---|---|
|
DNNL_ARG_SRC |
|
DNNL_ARG_DST |
workspace |
DNNL_ARG_WORKSPACE |
|
DNNL_ARG_DIFF_SRC |
|
DNNL_ARG_DIFF_DST |
Implementation Details
General Notes
During training, LRN might or might not require a workspace on forward and backward passes. The behavior is implementation specific. Optimized implementations typically require a workspace and use it to save some intermediate results from the forward pass that accelerate computations on the backward pass. To check whether a workspace is required, query the LRN primitive descriptor for the workspace. Success indicates that the workspace is required and its description will be returned.
Data Type Support
The LRN primitive supports the following combinations of data types:
Propagation |
Source / Destination |
|---|---|
forward / backward |
f32, bf16, f16 |
Data Representation
Source, Destination, and Their Gradients
Like most other primitives, the LRN primitive expects the following tensors:
Spatial |
Source / Destination |
|---|---|
0D |
|
1D |
|
2D |
|
3D |
|
The LRN primitive is optimized for the following memory formats:
Here, optimized^ means the format that comes out of any preceding compute-intensive primitive.
Post-ops and Attributes
The LRN primitive does not support any post-ops or attributes.
Implementation Limitations
Refer to Data Types for limitations related to data types support.
GPU
Supports only 2D spatial case.
Performance Tips
For backward propagation, use the same memory format for src, diff_dst, and diff_src (the format of the diff_dst and diff_src are always the same because of the API). Different formats are functionally supported but lead to highly suboptimal performance.
Example
This C++ API demonstrates how to create and execute a Local response normalization primitive in forward training propagation mode.