Performance Data for Intel® AI Data Center Products
Find the latest AI benchmark performance data for Intel Data Center products, including detailed hardware and software configurations.
Pretrained models, sample scripts, best practices, and tutorials
- Intel® Developer Cloud
- Intel® AI Reference Models and Jupyter Notebooks*
- AI-Optimized CPU Containers from Intel
- AI-Optimized GPU Containers from Intel
- Open Model Zoo for OpenVINO™ toolkit
- Jupyter Notebook tutorials for OpenVINO™
- AI Performance Debugging on Intel® CPUs
Measurements were taken using:
- PyTorch* Optimizations from Intel
- TensorFlow* Optimizations from Intel
- Intel® Distribution of OpenVINO™ Toolkit
Intel® Data Center GPU Flex Series 170
Deep Learning Inference
| Framework Version | Model | Dataset | Usage | Precision | Throughput | Batch size |
|---|---|---|---|---|---|---|
| Intel TensorFlow 2.13 | Stable Diffusion | Text Prompts | Generative AI | fp16 | 6.45 its/s | 1 |
| OpenVINO 2023.2 | ResNet50 v1.5 | ImageNet2012 | Image Recognition | int8 | 9,499 img/s | 256 |
| OpenVINO 2023.2 | ResNet50 v1.5 | ImageNet2013 | Image Recognition | int8 | 3,297 img/s | 1 |
| OpenVINO 2023.2 | Yolov 4 | COCO2017_detection | Object Detection | int8 | 1,209 img/s | 256 |
| OpenVINO 2023.2 | Yolov 4 | COCO2017_detection | Object Detection | int8 | 595 img/s | 1 |
| Intel® Pytorch 1.13 | ResNet50 v1.5 | ImageNet2012 | Image Recognition | int8 | 9,673 img/s | 1,024 |
| Intel® Pytorch 1.13 | Yolov4 | COCO2017_detection | Object Detection | int8 | 1,139 img/s | 256 |
| Intel TensorFlow 2.13 | ResNet50 v1.5 | ImageNet2012 | Image Recognition | int8 | 9,801 img/s | 1,024 |
| Intel TensorFlow 2.13 | EfficientNet-B0 | ImageNet2012 | Image Recognition | fp16 | 1,890 img/s | 128 |
| Intel TensorFlow 2.13 | EfficientNet-B3 | ImageNet2012 | Image Recognition | fp16 | 527 img/s | 128 |
| Intel TensorFlow 2.13 | MaskRCNN | COCO2017_detection | image Segmentation | fp16 | 30 img/s | 16 |
Hardware and software configuration (measured October 24, 2023):
Hardware configuration for Intel® Data Center GPU Flex Series 170 (code named Arctic Sound):Intel® Xeon® Platinum 8360Y CPU head node at 2.40 GHz, 2S-36C-72T, 128GB DDR4-3200 Memory, with 1 Intel® Data Center GPU Flex Series 170 card, Operating System: Ubuntu 22.04.