AI-Enabled Computer Vision Checkout Framework

A set of reference implementations and core services to help accelerate the development and deployment of AI-enabled Computer Vision solutions.

author-image

By

Overview

AI-enabled Computer Vision is reshaping the retail world, from the typical experiences at a cashier-operated Point of Sale (PoS) to state-of-the-art automated stores. The complexity of the solution, the application of computer vision, and the AI models used depend on the sophistication of the intended shopping experience.

The reference implementations and core services help accelerate the development and deployment of AI-enabled Computer Vision solutions.

Image showing the AI-Enabled Computer Vision Framework

You can sign up to know about the upcoming releases.

Reference Implementations

Reference Implementations are open-source sample solutions that provide critical components required to build and deploy the various AI-enabled computer vision use cases. These reference implementations are not meant to be a complete solution. They are an open-source initiative, providing evidence of hardware and software components needed to drive critical components of computer vision workloads.
 

*You can sign up to learn about the release of Shopping Cart and Autonomous Stores reference implementations.

Core Services

The core service modules in this framework are the foundation for building AI-enabled computer vision checkout experiences. Built-in core services support specific functional requirements and integrate the necessary frameworks and libraries to reduce the effort in building vertical use case references.

The core services provide:

  • Reusable vision pipeline for running any pre-trained deep learning model with encryption, decryption, preprocessing, and post-processing.
  • Optimized middleware, frameworks, and domain-specific microservices for faster solutions development.
  • Support for Go*, Python*, C* client libraries, and so on to help code easily.
  • Use case-specific pre-validated data and the flexibility to run your benchmarks. Combination of hardware and software options to build solutions with scalability and maintainability.
  • Choice of computing for the fixed functions to take advantage of diverse workload performance requirements.

Pipeline Management Service   

Bringing together a vision-enabled checkout system involves managing a vision pipeline and understanding the computing needs. Retailers will not run a single vision pipeline focused only on checkout workload but will have to run multiple workloads to ensure optimized use of their hardware. Hence, the reference implementations will look at fixed and flexible vision pipelines. 

The following figure shows the standard phases of a vision pipeline:

An image showing the standard phases of Pipeline Management Service
 

The pipeline includes several microservices that use the Intel hardware-optimized software ingredients to either distribute the workloads or deploy them in a single system. The roadmap showcases the features at a high level. 

Performance Evaluation Service  

The Performance Evaluation Service is an operating system agnostic tool that can be used for checkout reference use cases and can be tailored for other workloads to extract system metrics.

The Performance Evaluation Service is developed using the following open-source tools to derive the required metrics:

Metric Tools
CPU Utilization % System Activity Report (SAR), Sysstat
GPU Utilization % IGT (Integrated and Arc), Intel® XPU Manager (Flex GPU)
Compute Utilization % Intel® XPU Manager (Flex GPU)
Disk Read and Disk Read (MBs) Input/Output top
Power Draw (Watts) Intel® Performance Counter Monitor 
Memory Utilization % Free
Memory Bandwidth Usage (MBs) Free
Frames per Second Part of the Pipeline
Last Log Update Custom Pipeline
Total Text Count Custom Pipeline
Total Barcode Count Custom Pipeline

 

The code comprises a pipeline profile that you can configure before running the performance test. The profile allows you to define the environment variables, language, and other pertinent parameters. You can use this tool with Docker* containers to quickly deploy and benchmark workloads across Intel’s hardware portfolio.

The performance tests can be run across CPU, iGPU, and dGPU. The tests can be run across all architectures, in combination and individually, to provide flexibility in identifying performance needs.

You can run the script in two ways:

  • Defined Pipelines: This allows you to pre-define the number of pipelines and gather the performance metrics.
  • Stream Density: This allows you to pre-define the target frames per second (FPS) rate required per pipeline, and then measure metrics.   

All the metrics are captured in log files and will be available at the end of the performance run.


View Performance Data

Roadmap

Take a look at what will be available in the forthcoming releases.

Features:

  • Camera Management Service
  • Sensor Fusion Service
  • Deployment Service
  • Shopping Cart
  • Autonomous Stores

Resources:

  • Learning videos

To know about the future offerings of the Automated Self-Checkout Retail Reference Implementation, see the complete Roadmap on GitHub.