Intel® oneAPI Deep Neural Network Library Release Notes

Published: 12/05/2020  

Last Updated: 04/29/2022

By Hung-Ju Tsai, Jennifer L Jiang

This document provides a summary of new and changed product features.

Where to Find the Release

Please follow the steps to download Intel® oneAPI Base toolkit contained oneDNN from the Main Portal of Intel® oneAPI Base toolkit, and follow the installation instructions to install.

2022.1

Performance Optimizations

  • Intel® Processor Graphics and Xe architecture-based Graphics:
    • Improved performance for future Xe Architecture graphics (code name Ponte Vecchio).
    • Improved performance for future Arc graphics (code name Alchemist and DG2).
  • Intel® Architecture processors
    • Improved performance for future Intel Xeon Scalable processors (code name Sapphire Rapids). The functionality is now enabled by default and requires Linux kernel 5.16 or later.
    • Improved performance of matmul primitive for processors with Intel AVX-512 support.

New Functionality

  • Introduced bfloat16 destination support for int8 convolution, matmul and inner product primitives for processors with Intel AVX-512 support and or future Intel Xeon® Scalable processors (code name Sapphire Rapids)
  • Extended RNN primitive with support for AUGRU cell.
  • Added support for non-zero negative slope in ReLU post-op for batch normalization primitive.
  • Introduced support for mixed source and destination data types in softmax primitive.
  • Introduced persistent cache API. This functionality allows to serialize and reuse JIT kernels.

Usability

  • Reduced stack consumption in GEMM implementation.

Breaking Changes

  • Removed performance optimizations for Intel Xeon Phi processors. oneDNN will continue to be functional on these processors using Intel AVX2 codepath..

Deprecated Functionality

  • Support for SYCL 1.2.1 (aka SYCL 2017 standard) is deprecated and will be removed in future releases.

Known issues and limitations

Performance Optimizations

  • Intel® Processor Graphics and Xe architecture-based Graphics:
    • Introduced initial optimizations for future Xe Architecture graphics (code name Ponte Vecchio).
    • Improved pooling and layer normalization primitives performance.
  • Intel® Architecture processors
    • Improved performance for future Intel Xeon Scalable processors (code name Sapphire Rapids). The functionality is now enabled by default and requires Linux kernel 5.16.
    • Improved performance of matmul primitive for processors with Intel® Advanced Vector Extensions 512 (Intel® AVX-512) support.

New Functionality

  • Introduced support for compiler with SYCL 2020 standard support.
  • Introduced support for the ICX/ICPX and DPCPP compiler drivers available in the Intel® oneAPI DPC++ Compiler.

Usability

  • Added environment variables and build options with 'ONEDNN' prefix.

Breaking Changes

  • The Intel MKL-DNN compatibility API is removed. See Transition from Intel® MKL-DNN to oneDNN page for instructions on moving to the new API.

Deprecated Functionality

  • Support for Intel® Xeon Phi processors is deprecated and will be removed in the next release.
  • Support for SYCL 1.2.1 (aka SYCL 2017 standard) is deprecated and will be removed in future releases.

Known issues and limitations

Performance Optimizations

  • Improved primitive cache performance for Intel Graphics products.
  • Intel® Processor Graphics and Xe architecture-based Graphics:
    • Introduced initial optimizations for future Intel® Arc™ Graphics codenamed Alchemist (ACM). That includes optimizations of compute-bound primitives (Convolution, GEMM) for s8/u8, f16 and bf16 datatypes via DPAS (Dot Product Systolic Accumulate) instructions.
    • Improved performance of convolution and deconvolution after some OpenCL kernels were re-implemented using kernel code generator (jit:ir implementation as reported by DNNL_VERBOSE).
  • Intel® Architecture processors
    • Improved performance for future Intell® Xeon Scalable processor (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control.
    • Improved binary primitive performance for cases when one of the tensors is broadcasted.
    • Improved reorder primitive performance for memory formats with padding and/or zero points.
    • Improved performance of reduction primitive, reorder, shuffle primitives. 
    • Improved performance of depthwise forward convolution primitive for processors with Intel® AVX512 support.
    • Improved performance of forward inner product primitive for the shapes with minibatch equal to 1 for processors with Intel® AVX512 support.
    • Improved int8 GEMM performance for processors with Intell® AVX2 and Intel® DL Boost support.

New Functionality

  • Introduced PReLU post-op support in convolution and matmul.
  • Extended maximum allowed post-ops chain for compute primitives (convolution, deconvolution, inner product, and matmul) to 32.
  • Introduced support for zero points in sum post-op for convolution and matmul. The functionality is implemented only for CPUs.
  • Extended binary primitive with support for mixed data types for input tensors. The functionality is implemented only for CPUs.
  • Extended sum post-op for convolution and matmul primitives with support for mixed data types. The functionality is implemented only for CPUs.

Usability

  • Reduced overall library size by trimming down use of templates, OpenCL headers, and TBB headers. The configurations that benefitted the most are CPU only configuration with TBB threading.

Deprecated Functionality

  • Intel MKL-DNN compatibility API is deprecated and will be removed in the next update. See Transition from Intel MKL-DNN to oneDNN page for instructions on moving to new API.
  • Support for Intel Xeon Phi processors is deprecated and will be removed in the next release.

Known issues and limitations

Performance Optimizations

  • Extended primitive cache to improve primitive descriptor creation performance.
  • Improved primitive cache performance in multithreaded configurations.
  • Intel® Processor Graphics and Xe architecture-based Graphics:
    • Introduced initial optimizations for bfloat16 compute functionality for future Intel Xeon Scalable processor (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control.
    • Improved performance of binary primitive and binary post-op for cases with broadcast and mixed source and destination formats.
    • Improved performance of reduction primitive.
    • Improved performance of depthwise convolution primitive with NHWC activations for training cases
  • Intel® Architecture processors
    • Introduced initial optimizations for bfloat16 functionality for future Intel® Xeon Scalable processor with Intel® AMX support (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control.
    • Improved performance of int8 compute functionality for future Intel® Xeon Scalable processor (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control. 
    • Introduced initial performance optimizations for future Intel® Core processor with Intel® AVX2 and Intel® DL Boost instructions support (code name Alder Lake).
    • Improved performance of int8 primitives for processors with Intel® SSE4.1 instruction set support.
    • Improved performance of int8 and bfloat16 RNN and inner product primitives.
    • Introduced CPU ISA hints environment variable and API. New API is intended to dispatch function implementations using YMM registers to improve performance on processors with a single Intel® AVX512 compute unit.
    • Improved forward convolution performance for Intel® AVX-512 systems.
    • Improved convolution and batch normalization performance with threadpool.
    • Improved performance of bfloat16 shuffle primitive.
    • Improved performance of `dnnl_gemm` and functionality relying on this implementation for cases with `n=1` on all supported processors.
       

New Functionality

Usability

  • Introduced support for DPC++ debug configuration on Windows

Breaking changes

  • Updated minimal supported CMake version from to 2.8.12 (was 2.8.11)

Known issues and limitations

  • Backward inner product primitive may produce incorrect result for the shapes with number of output channels not been multiple by 16 for future Intel Xeon Scalable processor (code name Sapphire Rapids)
  • Convolution with binary post-op may produce incorrect results for formats with channel padding.
  • Pooling and batch normalization primitives may hang on Windows GEN9 and DG1 in DPC++/L0 configuration.
  • Pooling and batch normalization primitives with 4D double blocked memory formats may produce NaNs or hang on Linux DG1 platforms.
  • See DPC++ limitations that impact the library as well.

Performance Optimizations

  • Reduced overheads associated with primitive cache.
  • Intel® Processor Graphics and Xe architecture-based Graphics:
    • Improved performance of int8 primitives with NHWC activations format.
    • Improved functionality performance for padded memory formats.
    • Improved performance of reorder and shuffle primitives for multiple formats and all dimensions.
    • Improved performance of fp16 pooling primitive.
    • Improved performance of lnorm primitive for plain memory formats.
    • Improved performance of resampling primitive for blocked memory formats .
    • Improved performance of Winograd convolution.
  • Intel® Architecture processors
    • Introduced initial optimizations for bfloat16 functionality for future Intel® Xeon Scalable processor with Intel® AMX support (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control.
    • Improved performance of int8 compute functionality for future Intel® Xeon Scalable processor (code name Sapphire Rapids). The functionality is disabled by default and should be enabled via CPU dispatcher control. 
    • Introduced initial performance optimizations for future Intel® Core processor with Intel® AVX2 and Intel® DL Boost instructions support (code name Alder Lake).
    • Improved performance of int8 primitives for processors with Intel® SSE4.1 instruction set support.
    • Improved performance of int8 and bfloat16 RNN and inner product primitives.
    • Introduced CPU ISA hints environment variable and API. New API is intended to dispatch function implementations using YMM registers to improve performance on processors with a single Intel® AVX512 compute unit.
    • Improved forward convolution performance for Intel® AVX-512 systems.
    • Improved convolution and batch normalization performance with threadpool.
    • Improved performance of bfloat16 shuffle primitive.
    • Improved performance of `dnnl_gemm` and functionality relying on this implementation for cases with `n=1` on all supported processors.
       

New Functionality

  • Introduced binary post-op for (de)-convolution, pooling, eltwise, binary, inner product, matmul and reduction (GPU only) along with performance optimizations for CPUs and GPUs. Extended the number of supported post-ops for primitives to 20.
  • Extended eltwise primitive with support for `logsigmoid`, `mish`, `hardswish`, and `clip_v2` algorithms.
  • Introduced support for PRelu primitive
  • Introduced int8 support for LSTM primitive with projection for CPU.
  • Introduced asymmetric quantization support for int8 deconvolution.
  • Extended matmul implementation with support for per-output channel zero-points for quantization.
  • Extended support for broadcasting in binary primitive to both inputs for CPU.
  • Extended binary primitive with support for comparison operators.
  • Introduced float16 support in reduction primitive for GPU.
  • Introduced support for mixed input and output types in binary primitive for GPU.
  • Introduced support for post-ops in GPU resampling implementation.

Usability

  • Added API to enable displaying timestamps in oneDNN verbose mode. Timestamps allow to use oneDNN verbose output in profiling tools.
  • Improved presentation of oneDNN primitives in  Intel® VTune™ Profiler.

Validation

  • Extended benchdnn to report operation bandwidth.
  • Added ability to choose target GPU in benchdnn.

Known issues and limitations

  • When using driver version older than 27.20.100.9316 for Intel® UHD Graphics for 9th Gen Intel® Processor on Windows, convolution/de-convolution functions may sporadically hang or produce incorrect results in DPC++ configuration with LevelZero. Please upgrade your driver version to fix the issue. An alternative solution is to use DPC++ with OpenCL backend with DPC++ compiler.
  • Reorder, prelu, softmax, and pooling primitives on GPUs may be slower for zero padded memory formats than Intel oneDNN 2021.1.
  • Reorder operation for 5D tensor with two dimensions equal to 16 and one uneven dimension can produce incorrect results on Intel® Iris® Xe Max Graphics.
  • Eltwise primitive may produce incorrect results for oneDNN DPC++ configuration with Level Zero runtime. In order to avoid this, use DPC++ with OpenCL backend with DPC++ compiler.
  • Deconvolution primitive may segfault with int8 data on processors for cases with non-trivial padding on processors with Intel AVX-512 support.
  • Deconvolution primitive may segault with int8 data when used with post-ops and per_oc broadcast on processors with Intel AVX2 support.
  • Pooling, batch normalization, and binary primitives may segfault when executed on Xe architecture-based graphics. No workaround available.
  • Non-Intel GPUs are not supported. The library API allows to create a DNNL engine by index (the order of devices is determined by the SYCL runtime), and there is no check for GPU devices being non-Intel. To have more control, users can create a DNNL engine passing SYCL device and context explicitly.
  • When running GPU kernels that take longer than a certain time (it depends on OS and system settings), you may face a situation resulting in apparent hang of the application. There are ways to configure driver or system settings to disable this timeout and avoid hanging of DPC++ or OpenCL programs, including oneDNN examples:
  • See DPC++ limitations that impact the library as well.

New Functionality

Known issues and limitations

  • Pooling, batch normalization, and binary primitives may segfault when executed on Xe architecture-based graphics. No workaround available.
  • Non-Intel GPUs are not supported. The library API allows to create a DNNL engine by index (the order of devices is determined by the SYCL runtime), and there is no check for GPU devices being non-Intel. To have more control, users can create a DNNL engine passing SYCL device and context explicitly.
  • When running GPU kernels that take longer than a certain time (it depends on OS and system settings), you may face a situation resulting in apparent hang of the application. There are ways to configure driver or system settings to disable this timeout and avoid hanging of DPC++ or OpenCL programs, including oneDNN examples:
  • See DPC++ limitations that impact the library as well.

 

Notices and Disclaimers

Intel technologies may require enabled hardware, software or service activation.

No product or component can be absolutely secure.

Your costs and results may vary.

© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.