The vast accumulation of digital data requires new classes of applications that impact a computer user’s life. We are investigating computing platforms that can deliver enough performance for these future workloads to enable their use in mass-market applications. Recognition, Mining, and Synthesis (RMS) are three key classes of workloads that distill enormous amounts of data. Among these is Computer Vision (CV), an important workload that will greatly benefit from future architecture and algorithm innovations.

We illustrate these innovations by introducing and characterizing some of the most common CV algorithms and applications. We focus on (1) algorithms for Gaussian mixture models, (2) particle filtering (condensation filtering), and (3) optical flow/motion estimation, which are key ingredients of many modern CV algorithms. We also discuss computer vision applications, such as video surveillance, autonomous (intelligent) vehicles and driver assistance systems, entertainment and augmented reality, and smart health care.

We chose a complete video surveillance application as a representative case study for a complex CV workload. Video surveillance is one of the most resource-demanding CV applications that has wide-spread application. We analyze an entire pipeline of a video surveillance system to obtain computation and bandwidth characteristics.

Our characterization of individual CV algorithms as well as complete CV systems can be used to guide algorithm researchers to develop new algorithms that run faster on existing and future computing platforms. Furthermore, we hope that it will raise the awareness of the application developers to optimize their programs. It will also provide input data for architects to develop future computing platforms that run these workloads more efficiently.