Run GNMT FP32 Inference with a TensorFlow* Model Package

Published: 01/05/2021

Download Command



This document has instructions for running MLPerf* for GNMT FP32 inference using Intel® Optimization for TensorFlow*.

Download and unzip the MLPerf for GNMT model benchmarking data.

export DATASET_DIR=$(pwd)/nmt/data

Set the DATASET_DIR to point as instructed above when running MLPerf for GNMT.

Quick Start Scripts

Script name Description
fp32_online_inference Runs online inference (batch_size=1).
fp32_batch_inference Runs batch inference (batch_size=32).
fp32_accuracy Runs accuracy

Bare Metal

To run on bare metal, the following prerequisites must be installed in your environment:

  • Python* 3
  • Intel® Optimization for TensorFlow*
  • Numactl
  • TensorFlow add-ins - Instructions are provided later

After installing the prerequisites, download and untar the model package. Set environment variables for the path to your DATASET_DIR and an OUTPUT_DIR where log files will be written, then run a quick start script.

DATASET_DIR=<path to the dataset>
OUTPUT_DIR=<directory where log files will be written>

tar -xzf mlperf-gnmt-fp32-inference.tar.gz
cd mlperf-gnmt-fp32-inference

# TensorFlow addons (r0.5) build and installation instructions:
#   Clone TensorFlow addons (r0.5) and apply a patch: A patch file
#   is attached in Intel Model Zoo MLpref GNMT model scripts,
#   it fixes TensorFlow addons (r0.5) to work with TensorFlow 
#   version 2.3, and prevents TensorFlow 2.0.0 to be installed 
#   by default as a required dependency.

git clone --single-branch --branch=r0.5
cd addons
git apply ../models/language_translation/tensorflow/mlperf_gnmt/gnmt-v0.5.2.patch

#   Build TensorFlow addons source code and create TensorFlow addons
#   pip wheel. Use bazel 3.0.0 version :

bash  # answer yes to questions while running this script
bazel build --enable_runfiles build_pip_pkg
bazel-bin/build_pip_pkg artifacts
pip install artifacts/tensorflow_addons-*.whl --no-deps

cd .. # back to package dir
quickstart/<script name>.sh


Documentation and Sources

Get Started
Main GitHub*
Release Notes
Get Started Guide

Code Sources
Report Issue


License Agreement

LEGAL NOTICE: By accessing, downloading or using this software and any required dependent software (the “Software Package”), you agree to the terms and conditions of the software license agreements for the Software Package, which may also include notices, disclaimers, or license terms for third party software included with the Software Package. Please refer to the license file for additional details.

Related Containers and Solutions

GNMT FP32 Inference TensorFlow* Container

View All Containers and Solutions 🡢

Product and Performance Information


Performance varies by use, configuration and other factors. Learn more at