Transformative AI predicates on the ability of intelligent machines to reliably perceive, understand, and predict—even when sensory input signals are noisy, contradicting, incomplete or missing. To do this, AI systems must be able to measure their own uncertainty, as this will indicate the degree of confidence in the system’s output. This enables AI systems to understand their limitations, be transparent about them, and learn and adapt to imperfect or evolving operating conditions.
Probabilistic Computing research at Intel Labs focuses on system solutions that enable machines to quantify uncertainty for modeling, decision-making, and actuation. These capabilities open the door for more practical and impactful AI across many domains, including manufacturing, healthcare, robotics, autonomous driving, and more.
As humans, we subconsciously estimate uncertainty throughout the day and incorporate it into our decision making. Unlike current AI implementations, we intuitively understand that our choices or behavior do not guarantee absolute outcomes. We make the best choices we can based on past experiences, the environment, context, and other highly dynamic information. We estimate the probability of a traffic light turning red before we decide to pass through an intersection. If an object falls from a truck in front of us, our brain instantly calculates the most probable momentum of the object to inform our best evasive strategy. We constantly scan the environment, correlate it with what we already know, and use it to inform our actions so that we can safely reach our goals.
Probabilistic Computing seeks to mimic this level of intuition, prediction, and planning amid real-world uncertainty. To accomplish this goal, Intel Labs’ AI researchers are focused on (1) uncertainty quantification and calibration, (2) uncertainty-driven intuitive scene understanding, and (3) efficient probabilistic computing systems.
Uncertainty Quantification and Calibration
Today’s Deep Learning (DL) systems are trained to recognize patterns in visual, auditory, and other representations but cannot reliably indicate how confident or uncertain they are in their predictions. Deep neural networks (DNNs) are known to be overconfident when they make incorrect predictions when faced with data that is outside the distribution of their training data. These incorrect but overconfident predictions can be catastrophic in scenarios involving safety-critical systems such as robots or autonomous vehicles.
Intel Labs collaborates with external research labs across academia and industry to increase awareness of the role uncertainty plays in the evolution of truly intelligent AI. Our researchers are also collaboratively developing techniques that provide robust estimates of uncertainty.
To quantify uncertainty, Intel researchers are modeling probability distributions on parameters of deep neural networks (DNNs). Such probability distributions must be learned along with the DNN parameters during the training phase, which complicates the training process.1 Intel researchers have addressed this problem by developing a way to estimate and initialize these probability distributions before the full training[. This not only simplifies the subsequent training, but also improves the quality of the uncertainty. Intel researchers have also open-sourced Bayesian-Torch2., a library for Bayesian neural network layers and uncertainty estimation in Deep Learning, enabling the research/developer community to seamlessly build uncertainty-aware models.
A well-calibrated model should be accurate when it is certain about its prediction and indicate high uncertainty when it is likely to be inaccurate. Intel researchers have thus developed an optimization method that leverages the relationship between accuracy and uncertainty as an anchor for uncertainty calibration3. Our experiments demonstrate that this approach yields better model calibration than existing methods on large-scale image classification tasks.
For some scenarios where DNNs have already been deployed, training a new Bayesian neural network might not be possible. For such cases, Intel researchers have developed a measure of confidence for existing DNNs. Probability distributions are modeled on the outputs of one or more of the intermediate layers of a DNN4 instead of its parameters. Since the parameters of the DNN remain untouched, no network retraining is required. Further, the overhead of calculating the uncertainty score during actual operation is negligibly low. Our researchers have applied this method to the problem of industrial anomaly detection and localization and have achieved state-of-the-art results5.
Uncertainty-driven Intuitive Scene Understanding
For robust scene understanding in presence of noise, quantifying uncertainty is just the first step. The next step involves leveraging uncertainty in conjunction with perception to drive intuitive scene understanding. Modeling a scene along with the possible uncertainties allows an AI system to understand the environment it is operating in and predict the best action to achieve its goals.
Intel Labs researchers use probabilistic models to enable seamless collaboration between humans and autonomous agents. A robot working alongside a human operator on an assembly task can predict possible actions of the human collaborator and based on the least uncertain estimate, proceed to assist in the safest, most efficient and unintrusive manner. This cycle of perceive, estimate, predict and act is repeated in a continuous loop with the robot’s AI system constantly predicting and adjusting for uncertainty in the real-world operation.
Efficient Probabilistic Computing Systems
The training and inference of probabilistic computing workloads is memory and compute intensive. In Bayesian neural networks, for instance, models must learn probability distributions over the weights, which involves Monte Carlo sampling with multiple forward passes through the neural network. Inference algorithms need to run an ensemble of models, which results in higher compute and bandwidth requirements. Intel researchers are addressing these and other complexities with an optimized implementation of Bayesian Inference on a heterogenous platform that runs 200x faster than the traditional server platforms.
Probabilistic applications based on deep-feature modeling (DFM) have also been optimized to run highly efficiently on Intel platforms with Intel’s oneAPI ecosystem. Intel’s oneAPI ecosystem provides optimized and efficient implementations of popular AI libraries. These library functions are custom-coded for each target architecture, so no developer tuning is required when developers migrate code between different supported hardware architectures. It was demonstrated, for instance, that Intel Lab’s industrial anomaly detection algorithm was capable of running well over 300 fps on a single Xeon class processor, comfortably exceeding the typical frame-rate of 30 frames per second found on industrial cameras6.
A Sampling of Our Successes
Publications in top venues: Intel researchers have addressed several research problems in probabilistic computing including reliable uncertainty quantification; uncertainty calibration; efficient and optimized implementation; application to real-world problems such as industrial anomaly detection; continuous novelty detection and learning capable of operating in non-stationary conditions with recurrent exposure to new classes of data; and active learning. These have been published in top-tier AI/ML conferences contributing to the probabilistic deep learning community (see Additional Reading below).
Open-source contributions: Intel Labs has open-sourced Bayesian-Torch library to facilitate software developers and AI practitioners to build scalable uncertainty-aware models for real-world applications. Bayesian-Torch is designed to be flexible and enables seamless extension of standard deep neural network models to corresponding Bayesian deep models with simple APIs. We have also open-sourced our optimized, state-of-the-art anomaly detection solution which is available through AnomaLib, a ready-to-use anomaly detection library provided by Intel.
Product releases: Intel Labs uncertainty and feature-guided active learning algorithms are influencing Geti platform to speed up AI model development pipeline by reducing the data annotation efforts and the sampling biases in real-world use cases.
Next Steps
As Intel continues to develop and use probabilistic computing methods to improve current AI solutions and create new ones, we look forward to announcing improvements to AI-integrated uncertainty quantification, its applications to hardware design, autonomous systems, and various explainable/responsible and generative AI domains. In addition, our researchers are committed to making Intel the platform of choice for probabilistic computing and we will continue to make software and hardware enhancements available for efficient execution of these methods on our multiple platform offerings ranging from edge to client/server systems.
For More Information:
Intuitive and Efficient Human-robot Collaboration via Real-time Approximate Bayesian Inference
Uncertainty-aware Audiovisual Activity Recognition using Deep Bayesian Variational Inference
incDFM: Incremental Deep Feature Modeling for Continual Novelty Detection
Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical Bayes
Improving Model Calibration with Accuracy Versus Uncertainty Optimization
Improving MFVI in Bayesian Neural Networks with Empirical Bayes: A Study with Diabetic Retinopathy Diagnosis
Quantization of Bayesian Neural Networks and Its Effect on Quality of Uncertainty
Deep Probabilistic Models to Detect Data Poisoning Attacks
Continual Active Adaptation to Evolving Distributional Shifts
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Robust Contrastive Active Learning with Feature-guided Query Strategies
Sampling Methods Benchmark