MLPerf* 2023 Results: Intel's Amazing Performance on 4th Gen CPUs

MLPerf* is a benchmarking suite that measures the real-world performance of machine learning systems against a variety of machine learning tasks—image classification, object detection, machine translation, and others—in an architecture-neutral, representative, and reproducible manner.

This session focuses on Intel’s results from its 2023 MLPerf v.3.0 submissions to the data center category. The workload tasks were run on 4th gen Intel® Xeon® Scalable processors. (Fun fact: Intel remains the only data center CPU vendor to have MLPerf inference results on a broad set of models.)

Topics covered include:

• The remarkable gains of 4th gen versus 3rd gen Intel Xeon processors to run any AI workload, due in large part to specialized AI hardware accelerators like Intel® Advanced Matrix Extensions
• How additional joint submissions with other customers (all on 4th gen Intel Xeon processors) were competitive to NVIDIA* GPUs
• Key learnings to boost model performance on 4th gen Intel Xeon CPUs, such as platform configuration, memory balancing, and recommended BIOS settings (these can be checked using Intel® System Health Inspector)
• Methodologies and tools Intel used to optimize model performance for this submission
• Tips for how you can use MLPerf to benchmark your own model performance

Skill level: All

Download the following stand-alone versions:

• Intel® Extension for PyTorch*
• Intel® Neural Compressor
• Intel® oneAPI Deep Neural Network Library (oneDNN)

Explore the GitHub* repository for MLPerf v3.0 inference results from Intel.