Steps to Convert and Inference Detectron2* and Layout-LM* Models with OpenVINO™ Toolkit

Unable to determine how to convert Detectron2* and Layout-LM* models to OpenVINO™ Intermediate Representation (IR) and infer with CPU plugin.

To convert Detectron2 model, follow the Colab tutorial to train and inference the Detectron2* model with PyTorch* and also follow steps outlined below to convert model to ONNX* format and then to IR format. 

1. Export the Detectron2* PyTorch* model to ONNX* format by referring the official Detectron2, colab2 tutorials using the export_model.py with the tracing feature.

   $ python export_model.py --config-file ./output.yaml --sample-image ./new.jpg --output ./model/ --export-method tracing --format onnx MODEL.WEIGHTS ./output/model_final.pth MODEL.DEVICE cpu

2. For output.yaml, create the configuration file by adding the following lines in the Colab tutorial:

   print(cfg.dump())
   with open("output.yaml", "w") as f:
       f.write(cfg.dump())

3. Install OpenVINO via pip*:

   $ pip install openvino-dev

4. Convert ONNX* model to Intermediate Representation (IR):

   $ mo --input_model ./model/model.onnx

5. Run benchmark_app to inference with OpenVINO:

   With ONNX* model:
   $ benchmark_app -m model.onnx -d CPU -niter 100 -api async -hint throughput
   $ benchmark_app -m model.onnx -d CPU -niter 100 -api async -hint latency

   With IR model:
   $ benchmark_app -m model.xml -d CPU -niter 100 -api async -hint throughput
   $ benchmark_app -m model.xml -d CPU -niter 100 -api async -hint latency

To convert Layout-LM model. Follow steps outlined below to convert the Layout-LM model and inference with OpenVINO™.

1. Convert PyTorch* model to ONNX* model format:

   $ pip install transformers[onnx] 
   $ python3 -m transformers.onnx --model=microsoft/layoutlm-base-uncased onnx

2. Install OpenVINO via pip*:

   $ pip install openvino-dev 

3. Convert ONNX* model to Intermediate Representation (IR):

   $ mo --input_model ./onnx/model.onnx

4. Run benchmark_app to inference with OpenVINO:

   With ONNX* model:
   $ benchmark_app -m onnx/model.onnx -d CPU -niter 100 -api async -hint throughput -data_shape input_ids[1,1],bbox[1,1,4],attention_mask[1,1],token_type_ids[1,1]
   $ benchmark_app -m onnx/model.onnx -d CPU -niter 100 -api async -hint latency -data_shape input_ids[1,1],bbox[1,1,4],attention_mask[1,1],token_type_ids[1,1]

   With IR model:
   $ benchmark_app -m model.xml -d CPU -niter 100 -api async -hint throughput -data_shape input_ids[1,1],bbox[1,1,4],attention_mask[1,1],token_type_ids[1,1]
   $ benchmark_app -m model.xml -d CPU -niter 100 -api async -hint latency -data_shape input_ids[1,1],bbox[1,1,4],attention_mask[1,1],token_type_ids[1,1]

   Note	Input shapes (defined by -data_shape in above commands) may differ based on the usecase.