Disclaimer

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm

Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to: http://www.intel.com/products/processor_number/

BlueMoon, BunnyPeople, Celeron, Celeron Inside, Centrino, Centrino Inside, Core Inside, i960, Intel, the Intel logo, Intel Atom, Intel Atom Inside, Intel Core, Intel Inside, Intel Inside logo, Intel NetBurst, Intel NetMerge, Intel NetStructure, Intel SingleDriver, Intel SpeedStep, Intel Sponsors of Tomorrow., the Intel Sponsors of Tomorrow. logo, Intel StrataFlash, Intel Viiv, Intel vPro, Intel XScale, InTru, the InTru logo, InTru soundmark, Itanium, Itanium Inside, MCS, MMX, Moblin, Pentium, Pentium Inside, skoool, the skoool logo, Sound Mark, The Journey Inside, vPro Inside, VTune, Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.
* Other names and brands may be claimed as the property of others.

Microsoft, Windows, Visual Studio, Visual C++, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.

Java is a registered trademark of Oracle and/or its affiliates.

©Intel Corporation. All rights reserved.

Optimization Notice

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel.

Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804

License Definitions

By downloading and installing this product, you hereby agree that the accompanying materials are being provided to you under the terms and conditions of the End User License Agreement for the Intel® Integrated Performance Primitives (Intel® IPP) product previously accepted by you.

System Requirements

Recommended hardware:

Hardware requirements:

Software requirements:

For more information please see Intel® IPP System Requirements.

Overview

The data compression domain of the Intel® IPP library contains several functions that can speed up the well known LZ4 data compression method for both data compression and decompression operations.

For the detailed description of these functions, refer to the Volume 1: "Signal Processing/Data Compression Functions/Dictionary-Based Compression Functions" section of the Intel® IPP Developer Reference.

To use these functions, before building the LZ4 library you need to replace the original source code of several LZ4 functions with Intel® IPP functions calls.

This document provides instructions on how to prepare the LZ4 source code and build the library.

How to Build LZ4 with Intel® Integrated Performance Primitives Library

Prerequisites

Current dictionary contains source code patch file for LZ4 versions 1.7.5 and 1.8.1.2. Before building the library you need to download corresponding LZ4 source code archive from https://github.com/lz4/lz4/releases site.

Patching and Build on Linux* and macOS*

Download lz4-1.7.5.zip or lz4-1.8.1.2 file from LZ4 site to working directory.

Un-archive and patch the surce code using the following commands (it is assumed that patch file is in the working directory):

$ unzip lz4-1.7.5.zip
$ cd lz4-1.7.5
$ patch -p1 < ../lz4-1.7.5.patch.bin

or

$ unzip lz4-1.8.1.2.zip
$ cd lz4-1.8.1.2
$ patch -p1 < ../lz4-1.8.1.2.patch.bin

The output will be as follows:

patching file examples/Makefile
patching file lib/lz4.c
patching file lib/Makefile
patching file programs/Makefile
patching file tests/fuzzer.c
patching file tests/Makefile
patching file visual/VS2010/datagen/datagen.vcxproj
patching file visual/VS2010/frametest/frametest.vcxproj
patching file visual/VS2010/fullbench/fullbench.vcxproj
patching file visual/VS2010/fullbench-dll/fullbench-dll.vcxproj
patching file visual/VS2010/fuzzer/fuzzer.vcxproj
patching file visual/VS2010/liblz4/liblz4.vcxproj
patching file visual/VS2010/liblz4-dll/liblz4-dll.vcxproj
patching file visual/VS2010/lz4/lz4.vcxproj
patching file visual/VS2010/lz4.sln

Build the library using standard make utility

# Build the library and LZ4 tool
$ make WITH_IPP=yes ARCH=intel64

If you want to build LZ4 for IA-32 architecture specify "ARCH=ia32" in the "make" command line.

The above commands assume that IPPROOT environment variable points to head of Intel® IPP location.

Note
The conditional compilation statements, used for LZ4 source code files modification, allow to build original version of LZ4 from the same source code files. For that, you need to build LZ4 library using standard "make" command. If WITH_IPP definition not specified in compiler options, LZ4 source code files are compiled as unpatched.

Downloading, Patching and Build on Windows*

To download LZ4 source code files for Windows*, you can use the same URL addresses as for Linux*. However, the patching and building processes on Windows* are different.

Unpacking LZ4 archive

To get the LZ4 source code files on Windows* computer, you need to:

  • Download the .zip file from LZ4 site

  • Unpack the archive using any of available unzip tools for Windows*. For example, you can use 7-zip from http://www.7-zip.org/.

Patching Original LZ4 Files

There are no standard system tools for patching on Windows*, so you need to use 3-rd party tools. For example, http://gnuwin32.sourceforge.net/packages/patch.htm from Free Software Foundation.

With this tool you need to use the following command line options:

> patch.exe -p1 --binary < <patch file location>

Building LZ4 With Intel® IPP on Windows*

To build LZ4 library and tools on Windows*, there is Microsoft* Visual Studio* solution file and project files in <LZ4 package location>/visual/VS2010 directory. The patching procedure adds Intel® IPP-specific configurations Debug_IPP and Release_IPP to both win32 and x64 platforms.

These configurations are based on environment variable IPPROOT, which is set during Intel® IPP environment preparation.

LZ4 Basic Functions Modifed with Intel® IPP Function Calls

The current LZ4 source code patch file modifies the following set of LZ4 functionality:

LZ4 Function Additional Information

LZ4_compress_fast_extState

Intel® IPP is used if acceleration argument is equal to 0 or 1

LZ4_compress_destSize_extState

LZ4_compress_forceExtDict

LZ4_compress_fast

Calls LZ4_compress_fast_extState

LZ4_compress_destSize

Calls LZ4_compress_destSize_extState

LZ4_compress_limitedOutput_withState

Calls LZ4_compress_fast_extState. Obsolete function.

LZ4_compress_withState

Calls LZ4_compress_fast_extState. Obsolete function.

LZ4F_compressFrame

Intel® IPP function used for the 1-st frame

LZ4_decompress_safe

Performance Results

The following performance results obtained using LZ4 fullbench tool on LZ4 COPYING file from lz4-1.7.5/tests directory (file length 18092 bytes). CPU is Intel® Core™ i7-6700K at 4.00GHz.

LZ4 Function Original LZ4 Compressed Size Original LZ4 Ratio Original LZ4 Performance (MB/s) Modified LZ4 Compressed Size Modified LZ4 Ratio Modified LZ4 Performance (MB/s) Intel® IPP

LZ4_compress_default

10582

58.49%

517.1

10599

58.58%

725.9

Yes

LZ4_compress_default(small dst)

10582

58.49%

659.8

10599

58.58%

667.3

Yes

LZ4_compress_fast(0)

10582

58.49%

517.8

10599

58.58%

727.6

Yes

LZ4_compress_fast(1)

10582

58.49%

517.7

10599

58.58%

727.4

Yes

LZ4_compress_fast(2)

10951

60.53%

573.8

10951

60.53%

556.9

No

LZ4_compress_fast(17)

15583

86.13%

2320.1

15583

86.13%

2316.3

No

LZ4_compress_fast_extState(0)

10582

58.49%

516.3

10599

58.58%

724

Yes

LZ4_compress_fast_continue(0)

10647

58.85%

521.6

10647

58.85%

519.6

No

LZ4_compress_HC

9050

50.02%

75.1

9050

50.02%

75.5

No

LZ4_compress_HC_extStateHC

9050

50.02%

75.2

9050

50.02%

75.6

No

LZ4_compress_HC_continue

9050

50.02%

75.1

9050

50.02%

75.6

No

LZ4_compress_forceDict

10647

58.85%

620.8

10599

58.58%

607.5

Yes

LZ4F_compressFrame

10597

58.57%

652

10614

58.67%

662.1

Yes

LZ4_saveDict

18092

100.00%

1319857.3

18092

100.00%

1320734

No

LZ4_saveDictHC

18092

100.00%

1307290.5

18092

100.00%

1313457.4

No

LZ4_decompress_fast

2732

3824

Yes

LZ4_decompress_fast_usingDict

2767.6

3840.4

Yes

LZ4_decompress_safe

2620.1

4025.7

Yes

LZ4_decompress_safe_usingDict

2604

3631.4

Yes

LZ4_decompress_safe_partial

2568.7

3529.7

Yes

LZ4_decompress_safe_forceExtD

2497.4

3362.2

Yes

LZ4F_decompress

2464.2

3666.6

Yes

Technical Support

If you did not register your Intel® software product during installation, please do so now at the Intel® Software Development Products Registration Center. Registration entitles you to free technical support, product updates and upgrades for the duration of the support term.

For general information about Intel technical support, product updates, user forums, FAQs, tips and tricks, and other support questions, please visit (http://www.intel.com/software/products/support).

Note
If your distributor provides technical support for this product, please contact them rather than Intel.

For technical information about the Intel® IPP library, including FAQ’s, tips and tricks, and other support information, please visit the Intel® IPP forum: (http://software.intel.com/en-us/forums/intel-integrated-performance-primitives) and browse the Intel® IPP support page: https://software.intel.com/en-us/intel-ipp-support/.