Convolutional neural networks (CNNs) are powerful techniques for AI application development, offering the advantage of accuracy in image-recognition problems. In this talk, Intel software engineer Dmitry Matveev analyzes the performance and scalability of several software development tools that provide high-performance CNN-based, deep-learning inference on Intel® architectures.
Dmitry focuses on two typical data science problems: image classification1 and object detection2. His experiment plan is as follows:
- Prepare a set of trained models for several developer tools, including Intel® Distribution of OpenVINO™ toolkit, Intel® Optimization for Caffe*, and OpenCV.
- Select a large set of images from each dataset to ensure the performance analysis delivers accurate results; experimentally determine the most appropriate parameters (for example, batch size and the number of CPU cores used).
- Carry out computational experiments on Endeavor, NASA’s shared-memory supercomputer based on 2nd generation Intel® Xeon® Scalable processors (formerly code-named Cascade Lake).
With this experiment, this session covers:
- Performance of the Intel Distribution of OpenVINO toolkit, including comparing it to similar software for CNN-based deep-learning inference
- Analysis of the OpenVINO toolkit scaling efficiency using dozens of CPU cores in a throughput mode
- Results of the vector neural network instructions (VNNI) performance acceleration for Intel® Advanced Vector Extensions in Intel Xeon Scalable processors
- Analysis of modern CPU use in CNN-based deep-learning inference using the Roofline model included in Intel® Advisor
Check it out.
Download the Software
- Intel Distribution of OpenVINO Toolkit
- Intel Advisor stand-alone download (or get it as part of Intel® Parallel Studio XE or Intel® System Studio)
- Intel® Optimization for Caffe*
Software engineering manager, Intel Corporation
Dmitry focuses on deep-learning application development and optimization. Before joining Intel in 2016, he honed his software knowledge—from functional programming and object-oriented analysis and design to domain-specific languages, digital signal processing, and machine learning—at companies including MERA, SoftDrom, and Itseez. Dmitry holds an MA degree in computer science from Nizhniy Novgorod State Technical University.
1 Image Classification Model: ResNet50*; Dataset: ImageNET
2 Object Detection Model: SSD300; Dataset: PASCAL Visual Object Classes (VOC) 2012