In my role as head of artificial intelligence (AI) strategy at Intel, I’m often asked to provide background on the fundamentals of this rapidly advancing field. With that in mind, I’m beginning a series of “AI 101” posts to help explain the basics of AI. In this first post, I cover the relationship between AI, machine learning, and deep learning, as well as key factors fueling the current deep learning explosion.
AI refers to a broad class of systems that enable machines to mimic advanced human capabilities. AI use cases exist across every industry, and some of the most prevalent today (Figure 1) include:
● Image recognition – e.g., automated Facebook photo tagging
● Video classification – e.g., a security camera detecting a break-in
● Speech-to-text – e.g., dictating to your smartphone
● Natural language processing – e.g., text analytics, sentiment analysis, chatbots
● Recommendation systems – e.g., personalized advertising, product recommendations
● Tabular and time-series data applications – e.g., financial analysis, email spam filters, smart
wireless network routing
There are many ways to achieve AI. For example, by stringing together a long series of if/then statements and other rules, a programmer can create a so-called “expert system” that achieves the human-level feat of diagnosing a disease from symptoms. In machine learning, a machine automatically learns these rules by analyzing a collection of known examples. Machine learning is the most common way to achieve artificial intelligence today, and deep learning is a special type of machine learning. This relationship between AI, machine learning, and deep learning is shown in Figure 2.
Machine learning vs. deep learning
Let’s dig in a bit more on the distinction between machine learning and deep learning. Machine learning is a class of statistical methods that uses parameters from known existing data and then predicts outcomes on similar novel data. For example, given the history of home sales in a city, you could use machine learning to create a model that is able to predict how much a different home in that same city might sell for.
Traditionally, machine learning relies on a prescribed set of “features” that are considered important within the dataset. In our home-selling example, features relevant to a home’s price might be the number of bedrooms in the home, the size of the home in square feet, and standardized test scores in the school district. The process of building features into a machine learning algorithm is known as “feature engineering.” Feature engineering requires deep expertise in a given subject (here, residential real estate) and can be quite a labor-intensive process for the data scientist.
Deep learning is a type of machine learning that has received increasing focus in the last several years. With deep learning, the algorithm doesn’t need to be told about the important features. Instead, it is able to discover features from data on its own using a “neural network.” The name is inspired by a mathematical object called an artificial neuron that “fires” if the combination of inputs exceeds some threshold, just like a neuron in the brain does. Artificial neurons can be arranged in layers, and deep learning involves a “deep” neural network (DNN) that has many layers of artificial neurons.
Artificial neurons in a DNN are interconnected, and the strength of a connection between two neurons is represented by a number called a “weight”. The process of determining these weights is called “training” the DNN. We’ll cover DNN training in a future post.
An example: Face recognition
To better understand the distinction between machine learning and deep learning, consider a system designed to identify a person based on an image of their face (Figure 3).
In classic machine learning, a data scientist needs to identify the set of features that uniquely represent a given face -- for example, the roundness of the face or the distance between the eyes. Then you apply a machine learning classifier algorithm, which learns to associate a given pattern of features with a unique identity.
The difficulty with this approach is that it is often not known precisely what the useful features are for the problem in question. And even if we know that a feature is important, it may be hard to compute it. For example, in order to compute the distance between the eyes, you need to first be able to localize the eyes in the image, which in and of itself can be complicated. Now imagine trying to incorporate a feature such as hairstyle! We have a sense of what smoothed hair vs. parted hair vs. spiked hair may look like, but how do you define and measure this for use in an algorithm? Feature engineering can be extremely time consuming, and any inaccuracies in computing feature values will ultimately limit the quality of our results.
Deep learning enables us to avoid feature engineering altogether. Given enough “labeled data” (i.e., images of known faces) and the right tuning, a deep learning model will identify the most relevant features from the data on its own. Deep learning represents a conceptual shift in thinking, from "How do you engineer the best features?" to "How do you guide the model toward discovering the best features?”
Deep learning: Why now?
Many fundamental deep learning concepts have been around since the 1940s, but a number of recent developments have converged to supercharge the current deep learning revolution (Figure 4).
These developments include:
Bigger datasets – The scale of available data has increased dramatically, providing enough input to develop accurate models. For example, ImageNet is an open dataset of 10 million hand-labeled images, and Google's parent Alphabet has released eight million YouTube videos with category labels.
Better hardware – Training a typical deep learning model may require 10 exaflops (1018, or one quintillion, floating point operations) of compute. Due to Moore’s Law, hardware now exists that can perform this task cost- and time-effectively.
Smarter algorithms – Cloud service providers such as Google and Amazon have realized the value of AI and are investing heavily in fundamental research in the field. New algorithms are constantly being published, and most work is freely available.
The result has been an explosion of AI products and startups, and accuracy breakthroughs in image and speech recognition. Thanks to deep learning, machines now routinely demonstrate better than human-level accuracy (Figure 5). Deep learning is why Facebook is so good at recognizing who is in the photo you just uploaded and why Alexa generally gets it right when you ask her to play your favorite song.
Notices and Disclaimers
Intel technologies may require enabled hardware, software or service activation.
No product or component can be absolutely secure.
Your costs and results may vary.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.