Intel has been developing big data analytics frameworks and libraries built on Software Defined Infrastructure with open standard building blocks. From open enterprise-ready software platforms to analytics building blocks, runtime optimizations, tools, benchmarks, and use cases, Intel® software makes big data and analytics faster, easier, and more insightful. Examples include Apache Hadoop and Spark optimized frameworks, Intel Data Analytics Acceleration Libraries (Intel DAAL), and BigDL: Distributed Deep Learning on Apache Spark which runs over Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN).
These Intel frameworks and libraries are being integrated with Intel FPGA acceleration options. Customers can run unmodified applications which at run time can run on Intel Xeon®, or Intel FPGA, or other Intel platforms. Intel is also providing FPGA acceleration frameworks with end to end orchestration, virtualization, and security. Intel together with a partner ecosystem is offering unstructured, NoSQL, and relational data store acceleration with multi-function single Intel FPGAs, which accelerate data streams, networking, data access, and algorithms.
Traditional relational databases can benefit from significant acceleration with inline acceleration and protocol offload of networking, data streaming, and data access. Inline accelerators include compression, filtering, and encryption. The FPGA can also be used for memory access tasks such as cache management or memory mapped access. Indexing/lookups and filtering run very fast as FPGA’s excel at hashing and pattern matching with their flexible datapaths.
A strong requirement is that customer's SQL applications and database schemas should run without change. Swarm64 AS, an Intel partner, delivers a turnkey acceleration solution, Swarm64 SDA, for PostgreSQL, MariaDB, and MySQL databases. The Swarm64 SDA combines a PCIe* FPGA card with associated driver software for Linux* and a software plugin to the standard interfaces for those databases.
Swarm64 AS has demonstrated acceleration of high velocity data and real time analytics on PostgreSQL 9.6 using an Intel Xeon E5-2695v4 with 256GB of memory and an Intel Arria® 10-based PCIe card. Swarm64 AS achieved a more than 5X acceleration of data inserts and queries in the PostgreSQL 9.6 database accelerated by Swarm64 SDA versus a native PostgreSQL 9.6 database1. Swarm64 predicts that they will achieve about 2X overall acceleration of traditional data warehousing applications and over 3X storage compression1.
Intel is developing better compression for Hadoop/Spark reduce or “shuffle” phase with an approach which completely hides the FPGA by integrated to the Intel frameworks.
There are three additional opportunities for Spark acceleration:
Significant offloads are also available for NoSQL data stores. As one example, Intel partner AlgoLogic FPGA accelerates Key Value Store with networking protocol offload, local cache management, and fast lookups. AlgoLogic has demonstrated 3X messages per second at less than one tenth the latency. They also demonstrated that with FPGA acceleration, that the latency is predictable, compared to software-only KVS. This predictable latency is important when writing Service Level agreements to keep 99% latency below a certain value.
Data demands on IT continue to increase — from delivering high availability and managing storage to conducting near-real-time analytics. Relational databases and SQL continue to be the backbones for enterprise-class data analytics. Swarm64 offers an innovative add-on to PostgresSQL* that works with most common database and storage applications. It enables IT to handle large amounts of high-velocity data and helps eliminate the risks and costs inherent in introducing new IT systems. Most importantly, Swarm64SDA* is designed to significantly speed up data processing and analytics for demanding workloads. Swarm64SDA utilizes the latest generation Intel® FPGAs to deliver industry-leading performance for analytics use cases.
Swarm64SDA scales to many billions of rows and millions of inserts per second, helping to turn massive amounts of structured and unstructured data into actionable insight. It supports a wide spectrum of Internet of Things (IoT) capabilities, such as monitoring network connections, gathering sensor outputs from an array of intelligent devices, and ingesting behavioral data from customers or telemetry data from data centers or machine parks. Swarm64’s fast analytics on large, near-real-time datasets enable immediate action for use cases such as threat detection, customer conversion, and predictive maintenance.
The solution is easy to use and cost-effective, seamlessly adding onto most common databases including PostgreSQL, MariaDB*, and MySQL*.
Swarm64’s innovative approach is based on redesigning the data structures of the database and the data flow during processing, so that less data is touched during a query and data that needs to be processed moves faster through the system. The Intel FPGA increases I/O throughput and reduces the amount of data and the number of operations that the CPU needs to process. A software layer wraps the Intel FPGA processing and the proprietary data structures into an add-on that links to supported databases through standard interfaces.
The Swarm64 solution enables seamless cooperation between the CPU and the Intel FPGA, overcoming the latency increase and bandwidth limitations of storage accessed via the network or in a typical cloud infrastructure. This decouples storage from compute, enabling resource elasticity and an excellent cost-to-performance ratio.
Learn the benefits of the Swarm64 solutionSwarm64SDA – Seamless plug-inSwarm64 solution featuring Intel® Arria® 10 FPGAs
Tests measure performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.