Disclaimer

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm

Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to: http://www.intel.com/products/processor_number/

BlueMoon, BunnyPeople, Celeron, Celeron Inside, Centrino, Centrino Inside, Core Inside, i960, Intel, the Intel logo, Intel Atom, Intel Atom Inside, Intel Core, Intel Inside, Intel Inside logo, Intel NetBurst, Intel NetMerge, Intel NetStructure, Intel SingleDriver, Intel SpeedStep, Intel Sponsors of Tomorrow., the Intel Sponsors of Tomorrow. logo, Intel StrataFlash, Intel Viiv, Intel vPro, Intel XScale, InTru, the InTru logo, InTru soundmark, Itanium, Itanium Inside, MCS, MMX, Moblin, Pentium, Pentium Inside, skoool, the skoool logo, Sound Mark, The Journey Inside, vPro Inside, VTune, Xeon, and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.
* Other names and brands may be claimed as the property of others.

Microsoft, Windows, Visual Studio, Visual C++, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.

Java is a registered trademark of Oracle and/or its affiliates.

©Intel Corporation. All rights reserved.

Optimization Notice

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel.

Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

Notice revision #20110804

License Definitions

By downloading and installing this product, you hereby agree that the accompanying materials are being provided to you under the terms and conditions of the End User License Agreement for the Intel® Integrated Performance Primitives (Intel® IPP) product previously accepted by you.

System Requirements

Recommended hardware:

Hardware requirements:

Software requirements:

For more information please see Intel® IPP System Requirements.

Overview

The data compression domain of the Intel® IPP library contains several functions that can speed up the well known ZFP data compression method for both data compression and decompression operations. This is a library of lossy floating point compression functions developed by Peter Lindstrom from Lawrence Livermore National Laboratory (https://computation.llnl.gov/projects/floating-point-compression).

For the detailed description of these functions, refer to the Volume 1: "Signal Processing/Data Compression Functions/Floating Point Data Compression" section of the Intel® IPP Developer Reference.

To use these functions, before building the ZFP library you need to update the original source code of several ZFP functions with Intel® IPP functions calls.

This document provides instructions on how to prepare the ZFP source code and build the library.

Using Intel® IPP ZFP Functions Directly

Intel® IPP function interface for ZFP is simple and can be used directly as show in the example below:

#include <stdio.h>
#include <math.h>

#include <ippdc.h>
#include <ipps.h>

#define NX 100
#define NY 100
#define NZ 100

static double accuracy[] = {
    1.e-1, 1.e-2, 1.e-3, 1.e-4, 1.e-5, 1.e-6, 1.e-7
};
void InitSrcArray(Ipp32f*, int, int, int);
void Compress(const Ipp32f* pSrc, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp8u* pDst, int* pComprLen);
void Decompress(const Ipp8u* pSrc, int srcLen, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp32f* pDst);

int main()
{
    Ipp32f *pSrcArray, *pDstArray;
    Ipp8u *pBuffer;
    int accuracyIndex;
    int numFloats = NX * NY * NZ;

    pSrcArray = ippsMalloc_32f(numFloats);
    pDstArray = ippsMalloc_32f(numFloats);
    InitSrcArray(pSrcArray, NX, NY, NZ);
    pBuffer = ippsMalloc_8u(sizeof(Ipp32f) * numFloats);
    for (accuracyIndex = 0; accuracyIndex < sizeof(accuracy) / sizeof(double); accuracyIndex++)
    {
        int comprLen, i;
        double maxErr;

        Compress(pSrcArray, NX, NY, NZ, accuracy[accuracyIndex], pBuffer, &comprLen);
        printf("Accuracy = %-7g, ratio = %-5.2f, ", accuracy[accuracyIndex],
                                        (double)(sizeof(Ipp32f) * numFloats) / comprLen);
        Decompress(pBuffer, comprLen, NX, NY, NZ, accuracy[accuracyIndex], pDstArray);
        /* Absolute error calculation */
        maxErr = 0.;
        for (i = 0; i < numFloats; i++)
        {
            double locErr = fabs(pSrcArray[i] - pDstArray[i]);
            if (locErr > maxErr)
                maxErr = locErr;
        }
        printf("err = %-7.2g\n", maxErr);
    }
    ippsFree(pBuffer);
    ippsFree(pSrcArray); ippsFree(pDstArray);
}
/* Data initialization from ZFP's "simple" example */
void InitSrcArray(Ipp32f* pSrc, int dimX, int dimY, int dimZ)
{
    int i, j, k;

    for (k = 0; k < dimZ; k++)
        for (j = 0; j < dimY; j++)
            for (i = 0; i < dimX; i++) {
                double x = 2.0 * i / dimX;
                double y = 2.0 * j / dimY;
                double z = 2.0 * k / dimZ;
                pSrc[i + dimX * (j + dimY * k)] = (Ipp32f)exp(-(x * x + y * y + z * z));
            }
}

void Compress(const Ipp32f* pSrc, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp8u* pDst, int* pComprLen)
{
    int encStateSize;
    IppEncodeZfpState_32f* pEncState;
    int x, y, z;
    int yStep = maxY, zStep = maxX * maxY;

    ippsEncodeZfpGetStateSize_32f(&encStateSize);
    pEncState = (IppEncodeZfpState_32f*)ippsMalloc_8u(encStateSize);
    ippsEncodeZfpInit_32f(pDst, sizeof(Ipp32f) * (maxX * maxY * maxZ), pEncState);
    ippsEncodeZfpSetAccuracy_32f(accur, pEncState);
    for (z = 0; z < maxZ; z += 4)
        for (y = 0; y < maxY; y += 4)
            for (x = 0; x < maxX; x += 4)
            {
                const Ipp32f* pData = pSrc + x + y * yStep + z * zStep;
                ippsEncodeZfp444_32f(pData, yStep * sizeof(Ipp32f), zStep * sizeof(Ipp32f), pEncState);
            }
    ippsEncodeZfpFlush_32f(pEncState);
    ippsEncodeZfpGetCompressedSize_32f(pEncState, pComprLen);
    ippsFree(pEncState);
}
void Decompress(const Ipp8u* pSrc, int srcLen, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp32f* pDst)
{
    int decStateSize;
    IppDecodeZfpState_32f* pDecState;
    int x, y, z;
    int yStep = maxY, zStep = maxX * maxY;

    ippsDecodeZfpGetStateSize_32f(&decStateSize);
    pDecState = (IppDecodeZfpState_32f*)ippsMalloc_8u(decStateSize);
    ippsDecodeZfpInit_32f(pSrc, srcLen, pDecState);
    ippsDecodeZfpSetAccuracy_32f(accur, pDecState);
    for (z = 0; z < NZ; z += 4)
        for (y = 0; y < NY; y += 4)
            for (x = 0; x < NX; x += 4)
            {
                Ipp32f* pData = pDst + x + y * yStep + z * zStep;
                ippsDecodeZfp444_32f(pDecState, pData, yStep * sizeof(Ipp32f), zStep * sizeof(Ipp32f));
            }
    ippsFree(pDecState);
}

with the following output

Accuracy = 0.1    , ratio = 67.16, err = 0.0049
Accuracy = 0.01   , ratio = 41.22, err = 0.00088
Accuracy = 0.001  , ratio = 26.16, err = 0.00011
Accuracy = 0.0001 , ratio = 14.94, err = 8.9e-06
Accuracy = 1e-05  , ratio = 10.34, err = 1.4e-06
Accuracy = 1e-06  , ratio = 7.29 , err = 1.8e-07
Accuracy = 1e-07  , ratio = 4.68 , err = 3e-08

The above example is the simplest case when array dimensions are multiples of four. If not, you need to process data on the edges of source and destination arrays by compressing/decompressing using temporary 4x4x4 cube of 32-bit floating point values.

Another option of utilization of Intel® IPP functions for ZFP is using of open source ZFP library with preliminary source code patch with Intel® IPP function calls and build with Intel® IPP libraries.

How to Build ZFP Open Source Library with Intel® Integrated Performance Primitives Library

Prerequisites

Current dictionary contains source code patch files for ZFP versions 0.5.2, 0.5.4 and 0.5.5. Before building the library you need to download ZFP source code archive from any of the following sites:

Patching and Build on Linux* and macOS*

Download zfp-0.5.x.tar.gz (x is a neccesary version’s minor number) file from LLNL site to working directory.

Un-archive and patch the surce code using the following commands (it is assumed that zfp-0.5.x.patch file is in the working directory):

$ tar xf zfp-0.5.x.tar.gz
$ cd zfp-0.5.x
$ patch -p1 < ../zfp-0.5.x.patch.bin
patching file CMakeLists.txt
patching file Config
patching file examples/Makefile
patching file include/bitstream.h
patching file src/CMakeLists.txt
patching file src/inline/bitstream.c
patching file src/Makefile
patching file src/template/compress.c
patching file src/template/decompress.c
patching file tests/Makefile
patching file utils/Makefile

Build the library with standard make utility

$ make WITH_IPP=yes [ ARCH=ia32|intel64 ]

The option "WITH_IPP=yes" sets "-DWITH_IPP -I$IPPROOT/include" options in compiler command line. The option "ARCH=ia32|intel64" defines set of Intel® IPP libraries (32- or 64-bits) to be used in build of executables. By default, 64-bit libraries are used.

The above command assumes that IPPROOT environment variable points to the head of Intel® IPP location.

Note
The conditional compilation statements, used for ZFP source code files modification, allow to build original version of ZFP from the same source code files. For that, you need to build ZFP library using standard "make" command without Intel® IPP-related options. If WITH_IPP definition doesn’t exist in the compiler options, the ZFP source code files are compiled as unpatched.

The option "ZFP_WITH_OPENMP=1" enable support of threading mode via OpenMP for ZFP 0.5.4 and ZFP 0.5.5 versions.

Note
Threading with with OpenMP mode is’t supported in MacOSX.

Downloading, Patching and Build on Windows*

To download ZFP source code files for Windows*, you can use the same URL addresses as for Linux*. However, the patching and building processes on Windows* are different.

Unpacking ZFP Archive

To get the ZFP source code files on Windows* computer, you need to:

  • Download the .gz file from ZFP site

  • Unpack the archive using any of available unzip tools for Windows*. For example, you can use 7-zip from http://www.7-zip.org/.

Patching Original ZFP Files

There are no standard system tools for patching on Windows*, so you need to use 3-rd party tools. For example, http://gnuwin32.sourceforge.net/packages/patch.htm from Free Software Foundation.

With this tool you need to use the following command line options:

> patch.exe -p1 < <patch file location>

Building ZFP With Intel® IPP on Windows*

The ZFP main directory contains CMake script, which in turn is patched with Intel® IPP definitions. You may use this method as follows:

> mkdir build
> cd build
> cmake -A x64 -DWITH_IPP=yes ..
> msbuild ZFP.sln

The above is the example of preparing working space for of Microsoft* Visual Studio* IDE which was installed on the machine.

To build ZFP library for 32-bit architecture the following CMake command line is required:

> cmake -DWITH_IPP=yes -DARCH=ia32 ..

Technical Support

If you did not register your Intel® software product during installation, please do so now at the Intel® Software Development Products Registration Center. Registration entitles you to free technical support, product updates and upgrades for the duration of the support term.

For general information about Intel technical support, product updates, user forums, FAQs, tips and tricks, and other support questions, please visit (http://www.intel.com/software/products/support).

Note
If your distributor provides technical support for this product, please contact them rather than Intel.

For technical information about the Intel® IPP library, including FAQ’s, tips and tricks, and other support information, please visit the Intel® IPP forum: (http://software.intel.com/en-us/forums/intel-integrated-performance-primitives) and browse the Intel® IPP support page: https://software.intel.com/en-us/intel-ipp-support/.