Intel® Deep Learning Boost (Intel® DL Boost) - Improve Inference Performance of Hugging Face BERT Base Model in Google Cloud Platform (GCP) Technology Guide

Description

Malicious and phishing websites/emails detection is a big challenge for all network security companies. BERT is the best model to detect such attacks. It can provide very high accuracy. However, it takes longer inference time when compared to traditional methods. This technical solution shows how to take advantage of Intel AVX512/VNNI, oneDNN, and IPEX to boost AI Inference performance using Hugging Face BERT base model as an example. It will help the ISVs to adopt this inline inference solution into their NGFW/SASE. The evaluations were conducted on the GCP using three different hardware configurations.