Introduction to Digital Signage/Kiosk and Developing Display Technology

Digital displays and  kiosks are all around you.  You may have seen them cropping up at shopping centers and doctors’ offices. From video walls, to AR fitting mirrors, to ordering menus, to ticketing systems, digital signs are pervasive and are becoming a part of everyday shopping experience.

Types of Kiosks

There are different types of kiosks which include indoor and outdoor.

Indoor Kiosks

Indoor kiosks are connected, managed and secure self-service solutions that provide shoppers with engaging information. These are often extensions to the digital store experiences and leverage existing infrastructure and funded by the brand. This type of self-service solutions serve a variety of purposes in point of sale locations within Retail environments including self-checkout, product scanning, transactions, endless aisle, product look up and locator.

Outdoor Kiosks

Outdoor kiosks are connected, managed, and secure digital experiences in semi-outdoor and outdoor environments. These support informational and ad-funded digital networks and typically require additional infrastructure such as power, connectivity, outdoor rated equipment. This type of solution serves a wide variety of purposes in the Point of wait and Point of transit locations including streets, transit and venues providing value added services. 

Digital Signage vs Kiosks

In order to develop cutting edge display experiences, it’s important to understand the ecosystem and underlying motivations around digital signage.  The economics of digital signage is entirely based upon audience reach and influence and also the ability to attribute this reach and influence to a favorable outcome such as brand or product awareness or even an actual sale. Hence, the development motivation often centers on guaranteeing delivery of media content to an audience, gathering impression metrics, and reducing maintenance costs while increasing up time. 

The Modern Digital Sign

A state-of-the-art signage player consists of the following components:

• The media player is the central command of the digital sign.  It’s the primary driver of information to the audience.   It ultimately controls the advertising schedule and coordinates the decoding sequences that take media stored on a drive and display the content on the screen.  System integrators have a choice of several open source as well as for-pay content management systems for the media player, available across various operating systems and hardware configurations.  
• A camera is used to collect a data stream that is then processed for anonymous audience data. This data may include demographics, emotional reaction of viewers, and the number of impressions (people) the sign reached.  The process of extracting this useful information from a camera feed is called computer vision.
• Analytics consists of audience data and the data from the log of what has been played, then processing information to uncover trends and patterns. This process gives businesses insight into their audience.  It allows for tailoring of the message, as well as potentially adapting when and what the player chooses to play in order to have the most revenue impact.
• Security is necessary in a deployed environment to prevent hacking and malicious intrusions.  It is especially critical as more digital signs become connected to the internet and may be exposed to a wider array of attacks.
• When the player goes down, for example if the OS is locked up or the hardware becomes faulty, remote device management is the first line of defense to avoid a service call in the field.  It can also play an important role in optimizing your deployment, such as saving energy by turning off the signs at night or during closures.  Additionally, remote management allows for scalable updates for both the advertising content as well as player software.

Media Player Implementation

There are many ways to implement the software for a player. Understanding the various hardware components involved allows you to better leverage the software available to get what you want, whether it’s better performance or saving more energy.

The CPU is the consistent element across any Intel hardware.  All Intel processors will have a CPU with a common 64-bit architecture.  While you can perform all of the key functionalities of the player using the CPU (scheduling, decoding, playback, and computer vision), the CPU is really ideal for scheduling, running the operating system, doing administrative tasks (related to device management), security, and perhaps a small amount of analytics.

The GPU is usually better for higher quality rendering and shading of frames, as well as computer vision applications.  The GPU’s on the Intel® Core™ processor family range in capabilities from Intel® HD Graphics to Intel Iris™ graphics to Intel® Iris™ Pro graphics, in order of low to high capabilities.  

There is one often overlooked sub-block of the GPU, sometimes referred to as the Media Fixed Function block1, which contains hardware accelerators for encoding, decoding, transcoding, and certain video processing filters. The media block is critical to digital signage applications because it allows for offloading of decoding from the CPU and GPU’s cores.  For example the 7th Generation Intel® Processor family for S Platforms has these hardware accelerators for decoding of MPEG2, VC1/WMV9, AVC/H264, VP8, JPEG/MJPEG, and more [2].  In order to understand the capabilities of your particular processor, it’s best to check the datasheet.

The final component of the system is the display block, which drives the signal to the screen or monitor.

In order to understand how these all work together, below is a video analytics scenario showing how you would ideally use the various hardware components in combination with each other.

Video Analytics Example Data Flow

There are three main data flows to understand when implementing video analytics in digital signage.  

• The media flow plays the content on the screen.  
• The computer vision flow gets a data stream from a camera and processes it for anonymous audience data.  
• The analytics flow puts the media and computer vision flows together and extracts meaningful data.

An example of what this solution might looks like in its most basic form can be found in the RAS-100 Retail Analytics Solution Pack demo kit on the Intel Solutions Directory.

Media Flow

Going from left to right, the media starts out in a certain container format and is demuxed (basically split apart) by the CPU into audio, video, and metadata streams.  The video stream is decoded by the Media Fixed Function block into a bitstream format and then rendered with any metadata.  The rendered video is then put into a frame buffer that is in sync with the audio buffer.  

Computer Vision Flow

The camera sends an encoded stream (the format depends on the camera type you buy) to the CPU. The CPU unpacks the encoded stream and sends the video to the media block for decoding.  The information from the media block can then be used for computer vision processing by the CPU using the Intel® Distribution of OpenVINO™ toolkit or other computer vision libraries.  Decoding through the media block isn’t strictly necessary, but depends on the application, since some computer vision libraries can process encoded streams.

Analytics Flow

The timestamp data from the media flow and the results of the computer vision flow are combined.  Analysis can then be done on a range of scales.  At the local level, the player could potentially adapt the advertising based on the current live audience.  Either locally or on a server the data can be correlated for simple trends, such as which ads were seen the most, which had the most of a certain demographic, etc.  At the highest level, you can use machine learning and deep learning to infer understanding about why certain ads are performing, and predict what impact certain ads might have.  Tying all this information to ad campaigns or specific events can further increase value for advertisers.

SDK Options

Now that you understand a little more how digital signage works under the hood, there are several development options for you depending on what you’re doing:

Intel® Media SDK

The Intel® Media SDK allows for hardware acceleration of encoding, decoding, RAW photo and video processing, and certain video processing filters.  The Intel® Media SDK is the software API to employ when utilizing the underlying Media Fixed Function hardware.  However, it’s important to know that you’re coding to an optimized framework, not directly to hardware.  You should think of it more like a data stream or process that you set in motion than something you control step by step. 

There are samples and tutorials that come with the Intel® Media SDK and they serve different purposes.  The samples are ideal for command line feature tests (like installation verification) or evaluating new features.  The tutorials are simple starting points with coding examples.  So if it’s your first time using the SDK, you should start with the tutorials.

The Intel® Media SDK is available for Linux* client platforms as well in the form of the Intel® Media SDK for Embedded Linux. So whether your primary operating system is Windows* or Linux*, you can take advantage of these media processing capabilities.

For Windows client:

• Installation 
• Tutorials
• Developer Guide

Additional material:

• Get Amazing GPU Acceleration for Media Pipelines - Webinar Replay
• Framework for developing applications using Media SDK
• Integrating Intel® Media SDK with FFmpeg for mux/demuxing and audio encode/decode usages

Intel® Distribution of OpenVINO™ toolkit

Develop applications and solutions that emulate human vision with the Open Visual Inference & Neural Network Optimization (Intel® Distribution of OpenVINO™ toolkit) toolkit. Based on convolutional neural networks (CNN), the toolkit extends workloads across Intel® hardware and maximizes performance.

• Enables CNN-based deep learning inference on the edge
• Supports heterogeneous execution across computer vision accelerators—CPU, GPU, Intel® Movidius™ Neural Compute Stick, and FPGA—using a common API
• Speeds time to market via a library of functions and preoptimized kernels
• Includes optimized calls for OpenCV and OpenVX*

Intel® vPro™

The Intel® vPro™ Platform Solution Manager (Intel® vPro™ PSM) is a framework application that allows you to launch plugin applications to remotely manage your Intel vPro technology based PC clients.  Intel vPro PSM comes with several preconfigured plugins, which perform tasks such as power management, KVM Remote Control, and system defense, among others.

Intel SDK for OpenCL™ Applications

Intel SDK for OpenCL™ applications customizes heterogeneous compute applications and accelerates performance.

• Uniquely offload compute to Intel® Graphics Technology that's tailored to your specific application needs
• Deliver high-performance image and video processing pipelines
• Enhance power efficiency
• Take advantage of code builder functions for the OpenCL™ API to efficiently build, debug, and analyze kernel codes

Kiosk Displays