Accelerating and optimizing data analytics workflows has several challenges depending on your perspective and approach. Here are three examples:
- Database players view them from the perspective of storage, viewing analytics workload problems as an extension of database problems
- Higher-level programming languages and environments such as JVM result in tradeoffs between performance and ease of programming
- Data analytics workflows have been split between frameworks and tools that focus on analytic computation and those that focus on data visualization
In this highly informative talk, founder and CEO of OmniSci, Todd Mostak takes you on a comprehensive tour of how to use the latest high-performance computing (HPC) techniques to simultaneously accelerate analytics SQL and data visualization—a skill the company has been honing since 2013.
- Key lessons that OmniSci has learned by reexamining the nature of data-centric workflows, including how successive generations of hardware accelerators provide opportunities and unique technical challenges
- How these workloads can be viewed as a co-design problem that requires an understanding of the hardware and infrastructure characteristics, and the workload patterns themselves
- Techniques to accelerate analytic workflows by taking advantage of the hardware optimizations at every stage of the workflow—I/O acceleration to LLVM-based JIT compilation to large-scale, in-situ data visualization and efficient interfaces with machine-learning and deep-learning workflows
Get the Software
Get the Intel® oneAPI Base Toolkit, which includes many optimized tools and libraries for data analytics
Intel® Optane™ DC technology for data centers
Co-founder and CEO of OmniSci
Todd originally conceived the idea for OmniSci (formerly MapD Technologies) while conducting graduate research at Harvard on the Arab Spring. He later joined MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) in the database group as a research fellow, under the supervision of Professor Sam Madden, before founding MapD in late 2013.