“Accelerating with Purpose” for AI Everywhere


Naveen Rao

Artificial Intelligence (AI) is a transformative technology wave, allowing users in every industry to solve problems big and small. As a society, we’ll be able to tackle human challenges like clean water and stopping animal poaching while businesses can create new revenue streams and improve their bottom lines thanks to the limitless potential of AI. To get to this future state of “AI everywhere”, we’ll need to address the crush of data being generated and ensure enterprises are empowered to make efficient use of their data, processing it where it’s collected when it makes sense and making smarter use of their upstream resources. Data centers and the cloud need to have access to performant and scalable general purpose computing and specialized acceleration for complex AI applications. In this future vision of AI everywhere, a holistic approach is needed—from hardware to software to applications.

Holistic AI Hardware

In an AI empowered world, we will need to adapt hardware solutions into a combination of processors tailored to specific use cases – like inference processing at the edge –and customer needs that best deliver breakthrough insights. This means looking at specific application needs and reducing latency by delivering the best results as close to the data as possible.

At Intel, we believe that starts with 2nd Generation Intel® Xeon® Scalable processors, the only general-purpose processor with built-in AI acceleration in the form of Intel® DL Boost, which speeds up AI inference performance. We’ve also spent the last three years optimizing popular deep learning frameworks like TensorFlow*, Pytorch*, and PaddlePaddle* to take advantage of the AI instruction sets built directly into Intel Xeon scalable processors. For enterprises already running Intel Xeon Scalable processor-based data centers or cloud environments, general purpose computing offers a good foundation to start your AI journey.

Some of our customers have AI requirements beyond what they can achieve with a general-purpose CPU alone. Very-high throughput, constrained power, and high rack density are all reasons to look to alternative processing power for AI applications. To address these customer needs, Intel offers a range of AI processors and accelerators, ranging from products like the Intel® Movidius™ Myriad™ X Visual Processing Unit (VPU) to data-center accelerators like Intel® Arria®, Stratix® and Agilex™ FPGAs.

We’ve developed hardware solutions for the edge to the cloud that will let data scientists handle increasingly unstructured and complex data.

And at the 2019 Hot Chips conference, Intel will reveal new details of our upcoming high-performance AI accelerators – the Intel® Nervana™ Neural Network Processor-T (Intel® Nervana™ NNP-T) for training and the Intel® Nervana™ NNP-I for inference. To truly get to AI everywhere, we need dedicated accelerators like the Intel® Nervana™ NNPs, built from the ground up with a focus on AI. Only then will the industry be able to provide customers the right intelligence at the right time.

Intel® Nervana™ NNP-T

Intel Nervana NNP-T pushes the boundaries of deep learning training. It’s built to prioritize two key real-world considerations: how to train a network as fast as possible and how to do it within a given power budget. This deep learning training processor is also built with flexibility in mind and strikes a balance between compute, communication, and memory. Unlike a general-purpose CPU or a GPU, the architecture was built from the ground up, with no legacy workloads to support. And to account for future deep learning needs, the Intel Nervana NNP-T is built with flexibility and programmability in mind to accelerate the workloads of today as well as the workloads of tomorrow.

Intel® Nervana™ NNP-I

Intel Nervana NNP-I is purpose-built specifically for inference and is designed to accelerate deep learning deployment at scale, offering excellent performance for major data center workloads. Additionally, it offers a high degree of programmability without compromising performance to power efficiency. It’s easy to program, has short latencies, and fast code porting while supporting all major deep learning frameworks, allowing the world’s leading cloud service providers and enterprises to take advantage of advanced inference performance.

Tying it All Together with Software

It is critical to pair the hardware foundation for AI applications with a full software stack supporting all the latest frameworks. This ensures developers and data scientists will have the best real-world experience for applications. At Intel, we’ve optimized source libraries like nGraph, which supports training and inference across multiple frameworks and hardware architectures; developed the Intel® Distribution of OpenVINO™ toolkit to quickly optimize pretrained models and deploy neural networks for video to a variety of hardware architectures; and created BigDL*, our distributed deep learning library for Apache Spark* and Hadoop* clusters.

We’ve ensured that when new purpose-built hardware—like the Intel Nervana NNPs—is introduced, it integrates fully with existing developer tools and libraries to make the transition for developers and data scientists as seamless as possible.

The Intel Nervana NNP-T’s software stack is built with open components and supports direct integration with existing deep learning frameworks like TensorFlow*, PaddlePaddle*, and Pytorch* while the Intel Nervana NNP-I software stack supports all major deep learning frameworks and is highly programmable and flexible.

Learn More about the Intel® Nervana™ NNP Products

We’re excited to share even more details about the Intel Nervana NNP products at the upcoming Hot Chips conference – Intel speakers will be giving in-depth presentations about the capabilities and architecture of both the Intel Nervana NNP-T and the Intel Nervana NNP-I. Catch the Intel Nervana NNP-T (codenamed Spring Crest) presentation at 5:15 p.m. PT on August 19 and the Intel Nervana NNP-I (codenamed Spring Hill) presentation at 12:00 p.m. PT on Tuesday, August 20.

Follow @IntelAI and @IntelAIResearch on Twitter and visit the Intel AI page for more info on Intel AI technologies, tools and research.

Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available security updates. See configuration disclosures for details. No product or component can be absolutely secure.

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks.

Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804

Intel® Advanced Vector Extensions (Intel® AVX) provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware, software, and system configuration and you can learn more at http://www.intel.com/go/turbo.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.

Intel, the Intel logo, Xeon, and Nervana are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

*Other names and brands may be claimed as the property of others.

© Intel Corporation

Stay Connected

Keep tabs on all the latest news with our monthly newsletter.