Consider the following problem. You’re looking for new approaches to marketing a beloved 90s sitcom, which has already been enormously successful in syndication and home video sales. The show produced hundreds of half-hour episodes—more than 80 hours of video altogether. How do you determine which scenes will best resonate with your target audiences? How could you know what clips cue which emotions, and how those emotions might affect what people want to watch? How do you find the needle in the haystack that will maximize the impact of your ongoing marketing efforts?
Perhaps more importantly, where will you find time to review and carefully catalog dozens of hours of video?
Challenge: Curate Video Content with AI
I recently had the pleasure of sitting down with Warner Bros. Technology’s Justin Herz at O’Reilly AI NYC to discuss how he and his team are approaching these challenges with help from Intel. Herz is Warner Bros.’ Executive Vice President for Digital Product, Platform, and Strategy, and is responsible for implementing technology innovations that create new revenue, improve the bottom line, or transform how Warner Bros. produces, distributes, and markets its content.
For Herz, the earlier hypothetical use case is actually a real-world problem. In our talk, Herz spoke to the challenges Warner Bros. encountered when trying to analyze the more than 200-episode run of one of WB’s most celebrated television properties. This is an imposing amount of data. We later found, for example, that we needed to tag a minimum of 80,000 bounding boxes in order to be able to identify the main characters and their emotions. This data was then used to train up multiple models that work together to recognize talent, emotion, and intensity of emotion across all 10 seasons of the show.
In the past, Warner Bros. had attempted to create video metadata through various “black box” solutions that could programmatically extract metadata or manual tagging efforts that were difficult to scale. These earlier efforts were unsuccessful, and Warner Bros. turned to Intel.
Unprecedented Curation Efficiency with AI
Last year, Warner Bros. began working with Intel to use artificial intelligence to apply video metadata to improve content discovery and monetization. Initially, Warner Bros. worked on one of their properties before targeting to scale the solution across the rest of its content catalog.
To solve this problem, Intel data scientists first developed a training data set via crowdsourcing. The team then studied the training data—in Herz’ words, learning the “shape of the data”—to determine a suitable algorithm to be implemented.
After training and testing, the team selected a single shot multibox detector to identify talent and an ensemble of five convolutional neural networks for emotion and emotion intensity detection, running on Intel® Xeon® processors. Inference results were then stored in a database now used to identify clips in which characters demonstrate certain emotions. Warner Bros. is testing whether the impact of these emotions can drive intent to purchase, intent to view, and other business outcomes associated with marketing activities.
Using these technologies, Warner Bros. is becoming far more efficient at highlighting the clips that different core audiences most want to see and can do so on a scale impossible prior to applying AI to this problem. As a result, viewers can more easily connect with and purchase the content they most desire, increasing revenue for the studio.
To learn more about the possibilities AI can enable for your organization, please visit https://ai.intel.com.
Notices and Disclaimers
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.