# Intel® Integrated Performance Primitives Examples

# Disclaimer

## Legal Information

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm

Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to: http://www.intel.com/products/processor_number/

BlueMoon, BunnyPeople, Celeron, Celeron Inside, Centrino, Centrino Inside, Core Inside, i960, Intel, the Intel logo, Intel Atom, Intel Atom Inside, Intel Core, Intel Inside, Intel Inside logo, Intel NetBurst, Intel NetMerge, Intel NetStructure, Intel SingleDriver, Intel SpeedStep, Intel Sponsors of Tomorrow., the Intel Sponsors of Tomorrow. logo, Intel StrataFlash, Intel Viiv, Intel vPro, Intel XScale, InTru, the InTru logo, InTru soundmark, Itanium, Itanium Inside, MCS, MMX, Moblin, Pentium, Pentium Inside, skoool, the skoool logo, Sound Mark, The Journey Inside, vPro Inside, VTune, Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.
* Other names and brands may be claimed as the property of others.

Microsoft, Windows, Visual Studio, Visual C++, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.

Java is a registered trademark of Oracle and/or its affiliates.

©Intel Corporation. All rights reserved.

Optimization Notice
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #20110804

# License Definitions

By downloading and installing this product, you hereby agree that the accompanying materials are being provided to you under the terms and conditions of the End User License Agreement for the Intel® Integrated Performance Primitives (Intel® IPP) product previously accepted by you.

# System Requirements

Recommended hardware:

- A system based on 4th Generation Intel® Core™ i3, i5, i7 or newer processor

Hardware requirements:

- A system based on Intel® Quark™ System-on-Chip processors;

- A system based on Intel® Pentium® processor, Intel® Xeon® processor, or subsequent IA-32 architecture based system;

- A system based on a processor with Intel® 64 architecture.

Software requirements:

- For the basic system requirements on operating system, Intel® IPP library, IDE and compiler, refer to the corresponding Release Notes document.

For more information please see Intel® IPP System Requirements.

# Overview

The data compression domain of the Intel® IPP library contains several functions that can speed up the well known LZ4 data compression method for both data compression and decompression operations.

For the detailed description of these functions, refer to the Volume 1: "Signal Processing/Data Compression Functions/Dictionary-Based Compression Functions" section of the Intel® IPP Developer Reference.

To use these functions, before building the LZ4 library you need to replace the original source code of several LZ4 functions with Intel® IPP functions calls.

This document provides instructions on how to prepare the LZ4 source code and build the library.

# How to Build LZ4 with Intel® Integrated Performance Primitives Library

## Prerequisites

Current dictionary contains source code patch file for LZ4 versions 1.7.5 and 1.8.1.2. Before building the library you need to download corresponding LZ4 source code archive from https://github.com/lz4/lz4/releases site.

## Patching and Build on Linux* and macOS*

Download `lz4-1.7.5.zip` or `lz4-1.8.1.2` file from LZ4 site to working directory.

Un-archive and patch the surce code using the following commands (it is assumed that patch file is in the working directory):

    $ unzip lz4-1.7.5.zip
    $ cd lz4-1.7.5
    $ patch -p1 < ../lz4-1.7.5.patch.bin

or

    $ unzip lz4-1.8.1.2.zip
    $ cd lz4-1.8.1.2
    $ patch -p1 < ../lz4-1.8.1.2.patch.bin

The output will be as follows:

    patching file examples/Makefile
    patching file lib/lz4.c
    patching file lib/Makefile
    patching file programs/Makefile
    patching file tests/fuzzer.c
    patching file tests/Makefile
    patching file visual/VS2010/datagen/datagen.vcxproj
    patching file visual/VS2010/frametest/frametest.vcxproj
    patching file visual/VS2010/fullbench/fullbench.vcxproj
    patching file visual/VS2010/fullbench-dll/fullbench-dll.vcxproj
    patching file visual/VS2010/fuzzer/fuzzer.vcxproj
    patching file visual/VS2010/liblz4/liblz4.vcxproj
    patching file visual/VS2010/liblz4-dll/liblz4-dll.vcxproj
    patching file visual/VS2010/lz4/lz4.vcxproj
    patching file visual/VS2010/lz4.sln

Build the library using standard `make` utility

    # Build the library and LZ4 tool
    $ make WITH_IPP=yes ARCH=intel64

If you want to build LZ4 for IA-32 architecture specify "ARCH=ia32" in the "make" command line.

The above commands assume that `IPPROOT` environment variable points to head of Intel® IPP location.

Note | The conditional compilation statements, used for LZ4 source code files modification, allow to build original version of LZ4 from the same source code files. For that, you need to build LZ4 library using standard "make" command. If `WITH_IPP` definition not specified in compiler options, LZ4 source code files are compiled as unpatched.

## Downloading, Patching and Build on Windows*

To download LZ4 source code files for Windows*, you can use the same URL addresses as for Linux*. However, the patching and building processes on Windows* are different.

### Unpacking LZ4 archive

To get the LZ4 source code files on Windows* computer, you need to:

- Download the .zip file from LZ4 site

- Unpack the archive using any of available unzip tools for Windows*. For example, you can use `7-zip` from http://www.7-zip.org/.

### Patching Original LZ4 Files

There are no standard system tools for patching on Windows*, so you need to use 3-rd party tools. For example, http://gnuwin32.sourceforge.net/packages/patch.htm from Free Software Foundation.

With this tool you need to use the following command line options:

    > patch.exe -p1 --binary < <patch file location>

### Building LZ4 With Intel® IPP on Windows*

To build LZ4 library and tools on Windows*, there is Microsoft* Visual Studio* solution file and project files in `<LZ4 package location>/visual/VS2010` directory. The patching procedure adds Intel® IPP-specific configurations `Debug_IPP` and `Release_IPP` to both `win32` and `x64` platforms.

These configurations are based on environment variable `IPPROOT`, which is set during Intel® IPP environment preparation.

# LZ4 Basic Functions Modifed with Intel® IPP Function Calls

The current LZ4 source code patch file modifies the following set of LZ4 functionality:

LZ4 Function | Additional Information
--- | ---
`LZ4_compress_fast_extState` | Intel® IPP is used if `acceleration` argument is equal to 0 or 1
`LZ4_compress_destSize_extState` |
`LZ4_compress_forceExtDict` |
`LZ4_compress_fast` | Calls `LZ4_compress_fast_extState`
`LZ4_compress_destSize` | Calls `LZ4_compress_destSize_extState`
`LZ4_compress_limitedOutput_withState` | Calls `LZ4_compress_fast_extState`. Obsolete function.
`LZ4_compress_withState` | Calls `LZ4_compress_fast_extState`. Obsolete function.
`LZ4F_compressFrame` | Intel® IPP function used for the 1-st frame
`LZ4_decompress_safe` |

# Performance Results

The following performance results obtained using LZ4 `fullbench` tool on LZ4 `COPYING` file from `lz4-1.7.5/tests` directory (file length 18092 bytes). CPU is Intel® Core™ i7-6700K at 4.00GHz.

LZ4 Function | Original LZ4 Compressed Size | Original LZ4 Ratio | Original LZ4 Performance (MB/s) | Modified LZ4 Compressed Size | Modified LZ4 Ratio | Modified LZ4 Performance (MB/s) | Intel® IPP
--- | --- | --- | --- | --- | --- | --- | ---
LZ4_compress_default | 10582 | 58.49% | 517.1 | 10599 | 58.58% | 725.9 | Yes
LZ4_compress_default(small dst) | 10582 | 58.49% | 659.8 | 10599 | 58.58% | 667.3 | Yes
LZ4_compress_fast(0) | 10582 | 58.49% | 517.8 | 10599 | 58.58% | 727.6 | Yes
LZ4_compress_fast(1) | 10582 | 58.49% | 517.7 | 10599 | 58.58% | 727.4 | Yes
LZ4_compress_fast(2) | 10951 | 60.53% | 573.8 | 10951 | 60.53% | 556.9 | No
LZ4_compress_fast(17) | 15583 | 86.13% | 2320.1 | 15583 | 86.13% | 2316.3 | No
LZ4_compress_fast_extState(0) | 10582 | 58.49% | 516.3 | 10599 | 58.58% | 724 | Yes
LZ4_compress_fast_continue(0) | 10647 | 58.85% | 521.6 | 10647 | 58.85% | 519.6 | No
LZ4_compress_HC | 9050 | 50.02% | 75.1 | 9050 | 50.02% | 75.5 | No
LZ4_compress_HC_extStateHC | 9050 | 50.02% | 75.2 | 9050 | 50.02% | 75.6 | No
LZ4_compress_HC_continue | 9050 | 50.02% | 75.1 | 9050 | 50.02% | 75.6 | No
LZ4_compress_forceDict | 10647 | 58.85% | 620.8 | 10599 | 58.58% | 607.5 | Yes
LZ4F_compressFrame | 10597 | 58.57% | 652 | 10614 | 58.67% | 662.1 | Yes
LZ4_saveDict | 18092 | 100.00% | 1319857.3 | 18092 | 100.00% | 1320734 | No
LZ4_saveDictHC | 18092 | 100.00% | 1307290.5 | 18092 | 100.00% | 1313457.4 | No
LZ4_decompress_fast |  |  | 2732 |  |  | 3824 | Yes
LZ4_decompress_fast_usingDict |  |  | 2767.6 |  |  | 3840.4 | Yes
LZ4_decompress_safe |  |  | 2620.1 |  |  | 4025.7 | Yes
LZ4_decompress_safe_usingDict |  |  | 2604 |  |  | 3631.4 | Yes
LZ4_decompress_safe_partial |  |  | 2568.7 |  |  | 3529.7 | Yes
LZ4_decompress_safe_forceExtD |  |  | 2497.4 |  |  | 3362.2 | Yes
LZ4F_decompress |  |  | 2464.2 |  |  | 3666.6 | Yes

# Technical Support

If you did not register your Intel® software product during installation, please do so now at the Intel® Software Development Products Registration Center. Registration entitles you to free technical support, product updates and upgrades for the duration of the support term.

For general information about Intel technical support, product updates, user forums, FAQs, tips and tricks, and other support questions, please visit (http://www.intel.com/software/products/support).

Note | If your distributor provides technical support for this product, please contact them rather than Intel.

For technical information about the Intel® IPP library, including FAQ’s, tips and tricks, and other support information, please visit the Intel® IPP forum: (http://software.intel.com/en-us/forums/intel-integrated-performance-primitives) and browse the Intel® IPP support page: https://software.intel.com/en-us/intel-ipp-support/.