Summary
Why does the DETR model show different results when tested on different hardware?
Description
- Optimized DETR model with model optimization.
- Ran benchmark_app with the optimized model (DETR model) on the SPR machine and saved the latency and throughput.
- Ran the same benchmark_app and model on the Icelake machine and saved the latency and throughput.
- Both results showed significant differences.
Resolution
Below is the resolution after the details are examined.
- The accuracy difference in different hardware is expected behavior due to the memory and core of the processor.
- This can be supported by documentation where some comparisons were made on different hardware.
- For object detection, it is also expected to have multiple detections with different probabilities.
- In the object detection sample, the developer added a filter called confident_threshold to filter out detection with a lower probability.
- Suggestion to perform filter out the lower score in the code.