Inference Results Diverge When Compared with Local Environment and OpenVINO™ Model Server (OVMS) for the Same Model

• Used the Insurance Claim Processing repository and loaded YOLO Model to test the accuracy.
• Inference results diverge between the locally loaded YOLO ONNX model, and the same model loaded into OVMS.
• The results observed in 04-accident-recog were around 86%, whereas OVMS returned around 88%.

It appears that OpenVINO™ and Ultralytics loaded models received different inputs which causes the difference in the results.

A few changes need to be made to run in a local environment, these are the changes:

04-04-accident-recog.ipynb

1. Cell with reading input image and running prediction:

   original_image: np.ndarray = cv2.imread("images/carImage3.jpg")
   blob = cv2.dnn.blobFromImage(original_image, size=(640, 640), swapRB=False)
   blob = np.ascontiguousarray(blob[0].transpose((1,2,0)))
   results = model.predict(blob)

2. Cell that draws results on the original image:

   Image.fromarray(result.plot()[:,:,::-1].astype(np.uint8))

04-05-model-serving.ipynb

No changes in the notebook itself. Required changes in remote_infer.py.

1. Preprocess method:

   // code placeholder
   def preprocess(image_path):
   original_image: np.ndarray = cv2.imread(image_path)
   [height, width, _] = original_image.shape

   # Calculate scale factor
   scale = (height/640, width/640)

   # Preprocess the image and prepare blob for model
   blob = cv2.dnn.blobFromImage(original_image, scalefactor=1 / 255, size=(640, 640), swapRB=True)
   return blob, scale, original_image

2. Call to draw_bounding_box inside postprocess:

   draw_bounding_box(original_image, class_ids[index], scores[index], round(box[0] * scale[1]), round(box[1] * scale[0]),round((box[0] + box[2]) * scale[1]), round((box[1] + box[3]) * scale[0]))