AN 998: Intel® FPGA AI Suite Inference on Quantized Graphs

Download PDF
ID 786296
Date 12/01/2023
Public

1.2. Preparing a Quantized ResNet-18 Graph

To quantize the ResNet-18 graph with Jupyter* Notebooks, following these steps in terminal window session:
  1. Source the Python virtual environment with the installed packages:
    source ~/build-quantization-env/nncf_env/bin/activate
  2. Switch to the notebook directory:
    cd ~/build-quantization-env/openvino_notebooks/notebooks/\
305-tensorflow-quantization-aware-training
  3. Open the Jupyter* Notebook:
    jupyter lab
    Tip: If the browser does not open automatically, copy the link from the terminal session into your web browser. The link is typically something like http://localhost:8889/lab?token=<token_value>.
  4. In the Jupyter* Notebook, open 305-tensorflow-quantization-aware-training.ipynb and modify it as follows:
    1. Comment out or delete the first step:
      !pip-q install "tensorflow-datasets==4.2.0"
      The tensorflow-datasets package was installed earlier by the intall_ov_notebooks.sh script.
    2. Under Create and Initialize Quantization replace the following code block: 
      nncf_config_dict = {
    "input_info": {"sample_size": [1, 3] + list(IMG_SIZE)},
    "log_dir": str(OUTPUT_DIR),  # The log directory for NNCF-specific logging outputs.
    "compression": {
        "algorithm": "quantization",  # Specify the algorithm here.
    },
}

      with the following code block:

      nncf_config_dict = {
    "input_info": {"sample_size": [1, 3] + list(IMG_SIZE)},
    # The log directory for NNCF-specific logging outputs:
    "log_dir": str(OUTPUT_DIR),
    "target_device": "TRIAL",
    "compression": {
        "algorithm": "quantization",  # Specify the algorithm here.
        "preset": "performance",
        "quantize_inputs" : False,
        "quantize_outputs" : False,
        "weights" : {"mode":"symmetric","signed":True,},
        "activations"  :{"mode":"symmetric","signed":True,}
    },
}
  5. Run all the cells in the notebook.
  6. After you finish the notebook, you should see a folder called output on the left window of the notebook. The int8 ResNet-18 model in OpenVINO™ IR is in the output folder.

    In the file system, you can find OpenVINO™ IR files (ResNet-18_int8.xml and ResNet-18_int8.bin) under ~/build-quantization-env/openvino_notebooks/notebooks/305-tensorflow-quantization-aware-training/output/.

Level Two Title