Framework Version Model Usage Precision Throughput Perf/Watt Accuracy Latency(ms) Batch size
Intel PyTorch 1.13ResNet50 v1.5 Image Recognitionfp32640.06 img/s 76.13(%) with BS=128 1
Intel PyTorch 1.13ResNet50 v1.5Image Recognitionint82139.92 img/s 75.99(%) with BS=128 1
Intel PyTorch 1.13ResNet50 v1.5Image Recognitionfp32670.56 img/s   64
Intel PyTorch 1.13ResNet50 v1.5Image Recognitionint82414.29 img/s   116
Intel TensorFlow 2.11ResNet50 v1.5Image Recognitionfp32643.11 img/s 76.48(%) with BS=100 1
Intel TensorFlow 2.11ResNet50 v1.5Image Recognitionint82396.80 img/s 76.02(%) with BS=100 1
Intel TensorFlow 2.11ResNet50 v1.5Image Recognitionfp32663.59 img/s   64
Intel TensorFlow 2.11ResNet50 v1.5Image Recognitionint82723.73 img/s   116
OpenVINOResNet50 v1.5Image Recognitionfp32653.36 img/s 76.46(%) 1
OpenVINOResNet50 v1.5Image Recognitionint82517.58 img/s 76.36(%) 1
OpenVINOResNet50 v1.5Image Recognitionfp32666.39 img/s   64
OpenVINOResNet50 v1.5Image Recognitionint82679.54 img/s   116
Intel PyTorch 1.13BERTLarge SQuAD1.1 seq_len=384Natural Language Processingfp3217.42 sent/s 93.15(F1) with BS=8 1
Intel PyTorch 1.13BERTLarge SQuAD1.1 seq_len=384Natural Language Processingint868.52 sent/s 92.92(F1) with BS=8 1
Intel PyTorch 1.13BERTLarge SQuAD1.1 seq_len=384Natural Language Processingfp3219.77 sent/s   56
Intel PyTorch 1.13BERTLarge SQuAD1.1 seq_len=384Natural Language Processingint858.05 sent/s   56
Intel TensorFlow 2.11BERTLarge seq_len=384Natural Language Processingfp3219.24 sent/s 92.98(F1) with BS=32 1
Intel TensorFlow 2.11BERTLarge seq_len=384Natural Language Processingint844.20 sent/s 92.24(F1) with BS=32 1
Intel TensorFlow 2.11BERTLarge seq_len=384Natural Language Processingfp3219.00 sent/s   16
Intel TensorFlow 2.11BERTLarge seq_len=384Natural Language Processingint842.27 sent/s   16
OpenVINOBERTLargeNatural Language Processingfp3221.11 sent/s 93.25(F1) 1
OpenVINOBERTLargeNatural Language Processingint865.77 sent/s 92.65(F1) 1
OpenVINOBERTLargeNatural Language Processingfp3220.17 sent/s   16
OpenVINOBERTLargeNatural Language Processingint862.83 sent/s   16
Intel PyTorch 1.13SSD-ResNet34 COCO 2017 (1200 x1200)Object Detectionfp3215.04 img/s 20 mAP with BS=16 1
Intel PyTorch 1.13SSD-ResNet34 COCO 2017 (1200 x1200)Object Detectionint861.19 img/s 19.9 mAP with BS=16 1
Intel PyTorch 1.13SSD-ResNet34 COCO 2017 (1200 x1200)Object Detectionfp3214.95 img/s   112
Intel PyTorch 1.13SSD-ResNet34 COCO 2017 (1200 x1200)Object Detectionint857.64 img/s   112
Intel TensorFlow 2.11SSD-ResNet34Object Detectionfp3215.20 img/s 22.40 mAP 1
Intel TensorFlow 2.11SSD-ResNet34Object Detectionint860.93 img/s 21.40 mAP 1
Intel TensorFlow 2.11SSD-ResNet34Object Detectionfp3215.11 img/s   56
Intel TensorFlow 2.11SSD-ResNet34Object Detectionint859.70 img/s   56
OpenVINOSSD-ResNet34Object Detectionfp3276.85 img/s 20 mAP 1
OpenVINOSSD-ResNet34Object Detectionint8307.83 img/s 19.9 mAP 1
OpenVINOSSD-ResNet34Object Detectionfp3276.43 img/s   64
OpenVINOSSD-ResNet34Object Detectionint8317.44 img/s   64
Intel PyTorch 1.13RNNT LibriSpeechSpeech Recognitionfp3228.32 fps 7.31 WER with BS=64 1
Intel PyTorch 1.13RNNT LibriSpeechSpeech Recognitionfp32187.32 fps   448
Intel PyTorch 1.13ResNeXt101 32x16d ImageNetImage Classificationfp3277.49 fps 84.18(%) at BS=128 1
Intel PyTorch 1.13ResNeXt101 32x16d ImageNetImage Classificationint8268.56 fps 84.05(%) at BS=128 1
Intel PyTorch 1.13ResNeXt101 32x16d ImageNetImage Classificationfp3276.79 fps   64
Intel PyTorch 1.13ResNeXt101 32x16d ImageNetImage Classificationint8290.77 fps   116
OpenVINOResNeXt101 32x16d ImageNetImage Classificationfp3215.18 fps 84.17(%) 1
OpenVINOResNeXt101 32x16d ImageNetImage Classificationint861.68 fps 84.2(%) 1
OpenVINOResNeXt101 32x16d ImageNetImage Classificationfp3215.09 fps   64
OpenVINOResNeXt101 32x16d ImageNetImage Classificationint861.15 fps   64
Intel PyTorch 1.13MaskR-CNN COCO 2017Object Detectionfp3214.37 img/s   1
Intel PyTorch 1.13MaskR-CNN COCO 2017Object Detectionfp3212.39 img/s 37.82/34.23 bbox/segm 112
Intel PyTorch 1.13DLRM Criteo TerabyteRecommenderfp321062552.54 rec/s 80.27 AUC 128
Intel PyTorch 1.13DLRM Criteo TerabyteRecommenderint83712574.64 rec/s 80.24 AUC 128
Intel TensorFlow 2.11Transformer MLPerfLanguage Translationfp3212.06 sent/s 27.16 BLEU with BS=64 1
Intel TensorFlow 2.11Transformer MLPerfLanguage Translationint824.96 sent/s 27.11 BLEU with BS=64 1
Intel TensorFlow 2.11Transformer MLPerfLanguage Translationfp3260.40 sent/s   448
Intel TensorFlow 2.11Transformer MLPerfLanguage Translationint848.96 sent/s   448
Intel TensorFlow 2.11DIEN Amazon Books DataRecommenderfp3269547.37 rec/s 77.18(%) with BS=128 16
Intel TensorFlow 2.11DIEN Amazon Books DataRecommenderfp32236578.43 rec/s   65536
Intel TensorFlow 2.113D-UNet Image Segmentationfp321.45 samp/s 85.30 mean 1
Intel TensorFlow 2.113D-UNet Image Segmentationint83.63 samp/s 85.08 mean 1
Intel TensorFlow 2.113D-UNet Image Segmentationfp321.35 samp/s   6
Intel TensorFlow 2.113D-UNet Image Segmentationint83.30 samp/s   6
OpenVINO3D-UNet Image Segmentationfp321.44 samp/s 0.85 mean 1
OpenVINO3D-UNet Image Segmentationint84.91 samp/s 0.85 mean 1
OpenVINO3D-UNet Image Segmentationfp321.34 samp/s   6
OpenVINO3D-UNet Image Segmentationint84.47 samp/s   6