Anomaly Detection with Spiking Neural Networks

Published: 03/01/2021  

Last Updated: 03/01/2021

 

 

Introduction

In 2019, I had an opportunity to participate in the CERN openlab Summer Student Program. As part of this program, I designed a system of biologically plausible neural networks to classify elementary particles for the CMS experiment on the Large Hadron Collider. In 2020, we conducted further research in collaboration with Intel and Caltech to apply spiking neural networks (SNNs) to anomaly detection in time series.

I worked on two main aspects of the project during my internship with CERN in 2020:

  • Anomaly detection using spiking neural networks on Intel neuromorphic chip technology, code named Loihi
  • An inference engine for artificial neural networks with oneAPI and the hls4ml library

Our goal was to investigate the application of bio-inspired computing for real-time processing of data in high-energy physics. The work consisted of classification and regression problems, as well as anomaly detection for identification of new physics phenomena at CERN and LIGO.

Process

In our work, we proposed artificial neural networks using both unsupervised and supervised learning accelerated with oneAPI. This approach enables precise classification of familiar waveforms, as well as detection of exotic, unseen phenomena. On the experimental side, the study focused on possibilities for applying neuromorphic computing using various methods for conversion and direct training of spiking neural networks deployed on dedicated processors.

Intel® oneAPI Products Used

I was excited to come back to the research group at CERN and help to get oneAPI added to the existing technological stack. We worked closely with another CERN openlab online summer intern, Marcin Swiniarski, to enable the oneAPI back end in the custom hls4ml library. While Marcin designed the core architecture, I implemented new primitives, which enabled us to execute anomaly detection with hardware acceleration. Thanks to Data Parallel C++ (DPC++) representation of code, we are able to optimize and run efficient workloads on various Intel architectures using Intel® DevCloud. Currently the software is focused on CPU deployment but support for GPUs, FPGAs, or novel Intel® accelerators might be possible too.

Spiking neural networks are inspired by biological information processing, mimicking human brains. Their neurons contain the signal and activate at a certain threshold, incorporating the concept of time into their operation. This aspect makes them interesting candidates to be applied in scenarios with time-dependent data such as anomaly detection. There are many benefits and expectations from this technology. However, due to numerous challenges, specific software and hardware types are required for optimal performance. In the project, we worked on a dedicated neuromorphic processor using a Loihi technology-based system.

Results So Far

With our current results in gravitational wave detection using supervised learning for SNNs in Nengo, we achieved around 97% accuracy in simulation and 92% with on-chip deployment on Loihi technology. The results were compared with another method of conversion through SNN Toolbox, giving 91% accuracy for the model when run on the same hardware instance.

Precise information and benchmarks for the achieved results will be presented in the upcoming report summarizing the work.

Additional Resources

Project Details and Final Presentation

Project by Marcin Swiniarski: Inference Engine for Custom Neural Networks with oneAPI

Intel® DevCloud

About Bartlomiej Borzyszkowski

Bartlomiej Borzyszkowski is a master degree student of Control Engineering and Robotics at Gdansk University of Technology and is a deep learning software engineer at Intel. In 2019 and 2020, he participated in CERN openlab summer programs working on research projects devoted to the application of machine learning algorithms at the CMS Experiment. His scientific and professional interests are centered around artificial intelligence, decision systems, and robotics.

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.