Qlik Increases Big Data Analysis Speed by up to 29 Percent1 2

Qlik enables organizations to analyze disparate data sources using a visual interface. Performance is essential for usability. Qlik found that the Intel® Xeon® Platinum 8168 processor increased speed by up to 29 percent.

Qlik is a leading data analytics platform and a pioneer of user-driven business intelligence. Its portfolio of cloud-based and on-premise solutions meets customers’ growing needs from reporting and self-service visual analysis to guided, embedded and custom analytics, regardless of where data is located. Customers from many different industries use Qlik to gain meaning out of information from multiple sources, exploring the hidden relationships within data that lead to insights that ignite good ideas. Headquartered in Radnor, Pennsylvania, Qlik does business in more than 100 countries with over 40,000 customers globally.

Many businesses can feel overwhelmed by the amount of data they have, spread across many different sources. Qlik aims to solve that problem, by enabling companies to easily bring different data sources together. Its associative engine identifies links between the data, to enable it to be explored interactively.

“With traditional business intelligence, you typically have a static report,” says Frederic De Ranter, performance engineer, Qlik. “If you start making queries, you go down a linear path and start leaving data behind. You get an answer to the question you’re asking, but you don’t get the full picture. With Qlik’s solutions, when you make a selection, data is color coded to show the data in the selection in green, other data associated to it in white, and everything that is not associated in gray. With traditional tools, the gray would be invisible. Including it means that you can see not only your top sales in a particular month, but you could also easily find which items you’re not selling, shown in gray.”

The company’s portfolio consists of Qlik Sense*, its next generation analytics platform, web based utilizing HTML5. QlikView* is a guided analytics and dashboarding solution, and Qlik Sense* Cloud is the company’s Software as a Service (SaaS) offering. The solutions can be integrated with spreadsheets and other online tools, such as Salesforce*, and are designed to enable self-service analytics for business users across the organization.

A wide range of industries use Qlik. A sports retailer has used Qlik to create data dashboards to help it understand the impact of seasonal trends and customer loyalty. A UK city council uses Qlik solutions to help manage budgets and service levels to citizens. Its old business intelligence platform took seven days to run reports. Qlik now provides near-real-time data access. A healthcare provider uses Qlik to give clinicians insight into their own productivity, so they can make adjustments where necessary to earn performance-based bonuses. A whisky distillery monitors its production plant, and used the gray data to identify some machines that were left on, unused, overnight. Switching them off saved money in energy costs.

Performance matters
Performance is essential to enable users to explore their data intuitively. Data is cached in memory. As users make new selections, everything based on the selection is recalculated and the visualization is updated. If a page takes longer than a second or two to update, users will lose their train of thought, and their patience.

The speed of a particular implementation depends on three factors, which Qlik calls the “cornerstones of performance”:

  • The hardware the application is running on, including the server and its processor.
  • The application that the customer has built using Qlik. An application could be designed in such a way that its performance is not optimal.
  • The user scenario. How many users and what they are doing can affect performance and put a strain on the hardware. At the same time, if users are running similar queries, there is a greater likelihood of the data being cached in memory, which improves performance.

    For the hardware platform, Qlik provides a guide to the chipsets that the company has benchmarked and recommends. All of them are from Intel.

Testing the new Intel® Xeon® Platinum 8168 processor
Qlik worked with Intel to benchmark the performance of the new Intel® Xeon® Platinum 8168 processor, and compared its performance to the previous generation Intel® Xeon® processor E5-2699 v4, and the previous generation v3. The test used an internal Qlik benchmark that performs over 80 selections at the same time, simulating user interactions. The data set comprised 1 billion records of sales data, representing different customers in different countries. The calculations involved processing the data but excluding a single week from the data presented. Reports included the sales by year, the top ten customers with their sales total, and gross margins by product category shown using a treemap (a grid of proportionally sized boxes). The scenario stresses the processor’s CPU and memory.

Qlik’s customers place high demands on their compute resources…that’s why it is crucial to have a high-performance hardware platform with high-availability memory and a powerful processor. Companies that deploy Qlik on the latest Intel® Xeon® Platinum 8168 processor can offer their users a rich, user-friendly visual analytics platform that takes advantage of familiar productivity tools.

Anthony Deighton, CTO & SVP of Products

The same test code was run on each processor configuration. Qlik found that the new Intel® Xeon® Platinum 8168 processor delivered a 29 percent performance increase over the previous generation Intel® Xeon® processor E5-2699 v4, and a 63 percent increase over the earlier v3 processor (see benchmark chart).1

“With every generation, the calculation capacity of the processor has gone up,” says De Ranter. “This time, the core count increased from 22 to 24 cores. It’s not just about the core count, though: it’s also about how fast those cores are. Some customers load data every hour, and that process is single threaded. The new ability to increase the frequency of a particular core using Intel® Speed Shift Technology, while other cores continue to perform calculations at normal speed, will be a big help in this respect.”

He adds: “Since Qlik products are very dependent on the basic features (clock speed and core count) of a processor, and the interaction with memory, new technologies such as the Intel® Mesh Architecture and cache hierarchy improvements are very interesting to us.” The Intel® Mesh Architecture has been introduced in the Intel® Xeon® processor Scalable family, which includes the Intel® Xeon® Platinum processor. The architecture features an array of vertical and horizontal communication paths allowing traversal from one core to another through the shortest path, potentially improving core-to-cache and core-to-memory latency.

De Ranter adds: “Intel® Ultra Path Interconnect (Intel® UPI), new in the Intel® Xeon® processor Scalable family, offers a higher bandwidth between processors. This will make Qlik products perform better because all the DDR in the server is used and can now be accessed with lower response times.”

In developing and optimizing its software, Qlik has used  Intel® VTune™ Amplifier, which provides a visual interface for identifying bottlenecks in applications.

Conclusion
“Qlik’s customers place high demands on their compute resources, always pushing for more comprehensive analysis and discovery to gain new insights into business trends and opportunities,” says Anthony Deighton, CTO & SVP of Products, Qlik. “That’s why it is crucial to have a high-performance hardware platform with high-availability memory and a powerful processor. Companies that deploy Qlik on the latest Intel® Xeon® processor-based platforms can offer their users a rich, user-friendly visual analytics platform that takes advantage of familiar productivity tools.”

Product and Performance Information

1

Configuration information for Qlik’s tests: Software: QlikView*: HWE workload. OS: Windows Server* 2016. The workload is not I/O bound and is not memory size bound.
Baseline: 2S Intel® Xeon® processor E5-2699 v3, 2.3GHz, 18 cores, Intel® Turbo Boost Technology and Intel® Hyper-Threading Technology (Intel® HT Technology) enabled, BIOS 63.R00, 64GB total memory, 1600 MT/s / DDR4 LRDIMM, 1TB WDC SATA HDD.
Next gen: 2S Intel® Xeon® processor E5-2699 v4, 2.2GHz, 22 cores, Intel® Turbo Boost Technology and Intel® Hyper-Threading Technology (Intel® HT Technology) enabled, BIOS 63.R00, 64GB total memory, 1600 MT/s / DDR4 LRDIMM, 1TB WDC SATA HDD.
New: 2S Intel® Xeon® Platinum 8168 processor, 2.7 GHz, 24 cores, Intel® Turbo Boost Technology and Intel® Hyper-Threading Technology (Intel® HT Technology) enabled, BIOS 01.00.0412, 192GB total memory, 12 slots / 16GB / 2666 MT/s / DDR4 LRDIMM, 1 TB SATA HDD.
Testing by Intel and Qlik May 2017.

2

Benchmark results were obtained prior to implementation of recent software patches and firmware updates intended to address exploits referred to as "Spectre" and "Meltdown". Implementation of these updates may make these results inapplicable to your device or system. 

Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations, and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit http://www.intel.com/benchmarks.