A Performance & Scalability Analysis of CNN-Based Deep Learning Inference in the Intel® Distribution of OpenVINO™ Toolkit

@IntelDevTools

 

Convolutional neural networks (CNNs) are powerful techniques for AI application development, offering the advantage of accuracy in image-recognition problems. In this talk, Intel software engineer Dmitry Matveev analyzes the performance and scalability of several software development tools that provide high-performance CNN-based, deep-learning inference on Intel® architectures.

Dmitry focuses on two typical data science problems: image classification1 and object detection2. His experiment plan is as follows:

  1. Prepare a set of trained models for several developer tools, including Intel® Distribution of OpenVINO™ toolkit, Intel® Optimization for Caffe*, and OpenCV.
  2. Select a large set of images from each dataset to ensure the performance analysis delivers accurate results; experimentally determine the most appropriate parameters (for example, batch size and the number of CPU cores used).
  3. Carry out computational experiments on Endeavor, NASA’s shared-memory supercomputer based on 2nd generation Intel® Xeon® Scalable processors (formerly code-named Cascade Lake).

With this experiment, this session covers:

  • Performance of the Intel Distribution of OpenVINO toolkit, including comparing it to similar software for CNN-based deep-learning inference
  • Analysis of the OpenVINO toolkit scaling efficiency using dozens of CPU cores in a throughput mode
  • Results of the vector neural network instructions (VNNI) performance acceleration for Intel® Advanced Vector Extensions in Intel Xeon Scalable processors
  • Analysis of modern CPU use in CNN-based deep-learning inference using the Roofline model included in Intel® Advisor

Check it out.


Download the Software


Dmitry Matveev
Software engineering manager, Intel Corporation

Dmitry focuses on deep-learning application development and optimization. Before joining Intel in 2016, he honed his software knowledge—from functional programming and object-oriented analysis and design to domain-specific languages, digital signal processing, and machine learning—at companies including MERA, SoftDrom, and Itseez. Dmitry holds an MA degree in computer science from Nizhniy Novgorod State Technical University.

1 Image Classification Model: ResNet50*; Dataset: ImageNET
2 Object Detection Model: SSD300; Dataset: PASCAL Visual Object Classes (VOC) 2012

 

 

Intel® Distribution of OpenVINO™ Toolkit

Deploy deep learning inference with unified programming models and broad support for trained neural networks from popular deep learning frameworks.

Get It Now

See All Tools