What Is the Relationship between the Complexity of a Model and Its Inference Throughput?

Unable to determine why the Benchmark Tool reports similar inference FPS for two models of very different complexities, which is defined by Giga Floating Point Operations (GFLOPs).

• A higher number of FLOPs is directly proportional to a reduced number of FPS. However, there is no fair comparison between the FPS and FLOPs.
• Factors that can affect the FPS obtained from Benchmark Tool:
  • Run in synchronous and asynchronous mode
  • Number of streams
  • Number of inference requests

Intel® NCS2 is built on the Intel® Movidius™ Myriad™ X VPU featuring 16 programmable shave cores and a dedicated neural compute engine. This feature boosts up the performance of the networks.