Disclaimer
Legal Information
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS.
NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL
PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S
TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY
WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING
TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES
RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT
OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result,
directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY
SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES,
SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL
CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY,
ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL
APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING
OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time,
without notice. Designers must not rely on the absence or characteristics of any
features or instructions marked "reserved" or "undefined." Intel reserves these
for future definition and shall have no responsibility whatsoever for conflicts
or incompatibilities arising from future changes to them. The information here
is subject to change without notice. Do not finalize a design with this
information.
The products described in this document may contain design defects or errors
known as errata which may cause the product to deviate from published
specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest
specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this
document, or other Intel literature, may be obtained by calling 1-800-548-4725,
or go to:
http://www.intel.com/design/literature.htm
Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. Go to: http://www.intel.com/products/processor_number/
BlueMoon, BunnyPeople, Celeron, Celeron Inside, Centrino, Centrino Inside, Core Inside,
i960, Intel, the Intel logo, Intel Atom, Intel Atom Inside, Intel Core,
Intel Inside, Intel Inside logo, Intel NetBurst, Intel NetMerge,
Intel NetStructure, Intel SingleDriver, Intel SpeedStep,
Intel Sponsors of Tomorrow., the Intel Sponsors of Tomorrow. logo,
Intel StrataFlash, Intel Viiv, Intel vPro, Intel XScale, InTru, the InTru logo,
InTru soundmark, Itanium, Itanium Inside, MCS, MMX, Moblin, Pentium,
Pentium Inside, skoool, the skoool logo, Sound Mark, The Journey Inside,
vPro Inside, VTune, Xeon, and Xeon Inside
are trademarks of Intel Corporation in the U.S. and other countries.
* Other names and brands may be claimed as the property of others.
Microsoft, Windows, Visual Studio, Visual C++, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.
Java is a registered trademark of Oracle and/or its affiliates.
©Intel Corporation. All rights reserved.
Optimization Notice |
---|
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not
unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and
other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of
any optimization on microprocessors not manufactured by Intel.
Notice revision #20110804 |
License Definitions
By downloading and installing this product, you hereby agree that the accompanying materials are being provided to you under the terms and conditions of the End User License Agreement for the Intel® Integrated Performance Primitives (Intel® IPP) product previously accepted by you.
System Requirements
Recommended hardware:
-
A system based on 4th Generation Intel® Core™ i3, i5, i7 or newer processor
Hardware requirements:
-
A system based on Intel® Quark™ System-on-Chip processors;
-
A system based on Intel® Pentium® processor, Intel® Xeon® processor, or subsequent IA-32 architecture based system;
-
A system based on a processor with Intel® 64 architecture.
Software requirements:
-
For the basic system requirements on operating system, Intel® IPP library, IDE and compiler, refer to the corresponding Release Notes document;
-
CMake version 3.1 or higher for CMake-based builds.
For more information please see Intel® IPP System Requirements.
Overview
The data compression domain of the Intel® IPP library contains several functions that can speed up the well known ZFP data compression method for both data compression and decompression operations. This is a library of lossy floating point compression functions developed by Peter Lindstrom from Lawrence Livermore National Laboratory (https://computation.llnl.gov/projects/floating-point-compression).
For the detailed description of these functions, refer to the Volume 1: "Signal Processing/Data Compression Functions/Floating Point Data Compression" section of the Intel® IPP Developer Reference.
To use these functions, before building the ZFP library you need to update the original source code of several ZFP functions with Intel® IPP functions calls.
This document provides instructions on how to prepare the ZFP source code and build the library.
Using Intel® IPP ZFP Functions Directly
Intel® IPP function interface for ZFP is simple and can be used directly as show in the example below:
#include <stdio.h> #include <math.h> #include <ippdc.h> #include <ipps.h> #define NX 100 #define NY 100 #define NZ 100 static double accuracy[] = { 1.e-1, 1.e-2, 1.e-3, 1.e-4, 1.e-5, 1.e-6, 1.e-7 }; void InitSrcArray(Ipp32f*, int, int, int); void Compress(const Ipp32f* pSrc, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp8u* pDst, int* pComprLen); void Decompress(const Ipp8u* pSrc, int srcLen, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp32f* pDst); int main() { Ipp32f *pSrcArray, *pDstArray; Ipp8u *pBuffer; int accuracyIndex; int numFloats = NX * NY * NZ; pSrcArray = ippsMalloc_32f(numFloats); pDstArray = ippsMalloc_32f(numFloats); InitSrcArray(pSrcArray, NX, NY, NZ); pBuffer = ippsMalloc_8u(sizeof(Ipp32f) * numFloats); for (accuracyIndex = 0; accuracyIndex < sizeof(accuracy) / sizeof(double); accuracyIndex++) { int comprLen, i; double maxErr; Compress(pSrcArray, NX, NY, NZ, accuracy[accuracyIndex], pBuffer, &comprLen); printf("Accuracy = %-7g, ratio = %-5.2f, ", accuracy[accuracyIndex], (double)(sizeof(Ipp32f) * numFloats) / comprLen); Decompress(pBuffer, comprLen, NX, NY, NZ, accuracy[accuracyIndex], pDstArray); /* Absolute error calculation */ maxErr = 0.; for (i = 0; i < numFloats; i++) { double locErr = fabs(pSrcArray[i] - pDstArray[i]); if (locErr > maxErr) maxErr = locErr; } printf("err = %-7.2g\n", maxErr); } ippsFree(pBuffer); ippsFree(pSrcArray); ippsFree(pDstArray); } /* Data initialization from ZFP's "simple" example */ void InitSrcArray(Ipp32f* pSrc, int dimX, int dimY, int dimZ) { int i, j, k; for (k = 0; k < dimZ; k++) for (j = 0; j < dimY; j++) for (i = 0; i < dimX; i++) { double x = 2.0 * i / dimX; double y = 2.0 * j / dimY; double z = 2.0 * k / dimZ; pSrc[i + dimX * (j + dimY * k)] = (Ipp32f)exp(-(x * x + y * y + z * z)); } } void Compress(const Ipp32f* pSrc, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp8u* pDst, int* pComprLen) { int encStateSize; IppEncodeZfpState_32f* pEncState; int x, y, z; int yStep = maxY, zStep = maxX * maxY; ippsEncodeZfpGetStateSize_32f(&encStateSize); pEncState = (IppEncodeZfpState_32f*)ippsMalloc_8u(encStateSize); ippsEncodeZfpInit_32f(pDst, sizeof(Ipp32f) * (maxX * maxY * maxZ), pEncState); ippsEncodeZfpSetAccuracy_32f(accur, pEncState); for (z = 0; z < maxZ; z += 4) for (y = 0; y < maxY; y += 4) for (x = 0; x < maxX; x += 4) { const Ipp32f* pData = pSrc + x + y * yStep + z * zStep; ippsEncodeZfp444_32f(pData, yStep * sizeof(Ipp32f), zStep * sizeof(Ipp32f), pEncState); } ippsEncodeZfpFlush_32f(pEncState); ippsEncodeZfpGetCompressedSize_32f(pEncState, pComprLen); ippsFree(pEncState); } void Decompress(const Ipp8u* pSrc, int srcLen, int maxX, int maxY, int maxZ, Ipp64f accur, Ipp32f* pDst) { int decStateSize; IppDecodeZfpState_32f* pDecState; int x, y, z; int yStep = maxY, zStep = maxX * maxY; ippsDecodeZfpGetStateSize_32f(&decStateSize); pDecState = (IppDecodeZfpState_32f*)ippsMalloc_8u(decStateSize); ippsDecodeZfpInit_32f(pSrc, srcLen, pDecState); ippsDecodeZfpSetAccuracy_32f(accur, pDecState); for (z = 0; z < NZ; z += 4) for (y = 0; y < NY; y += 4) for (x = 0; x < NX; x += 4) { Ipp32f* pData = pDst + x + y * yStep + z * zStep; ippsDecodeZfp444_32f(pDecState, pData, yStep * sizeof(Ipp32f), zStep * sizeof(Ipp32f)); } ippsFree(pDecState); }
with the following output
Accuracy = 0.1 , ratio = 67.16, err = 0.0049
Accuracy = 0.01 , ratio = 41.22, err = 0.00088
Accuracy = 0.001 , ratio = 26.16, err = 0.00011
Accuracy = 0.0001 , ratio = 14.94, err = 8.9e-06
Accuracy = 1e-05 , ratio = 10.34, err = 1.4e-06
Accuracy = 1e-06 , ratio = 7.29 , err = 1.8e-07
Accuracy = 1e-07 , ratio = 4.68 , err = 3e-08
The above example is the simplest case when array dimensions are multiples of four. If not, you need to process data on the edges of source and destination arrays by compressing/decompressing using temporary 4x4x4 cube of 32-bit floating point values.
Another option of utilization of Intel® IPP functions for ZFP is using of open source ZFP library with preliminary source code patch with Intel® IPP function calls and build with Intel® IPP libraries.
How to Build ZFP Open Source Library with Intel® Integrated Performance Primitives Library
Prerequisites
Current dictionary contains source code patch files for ZFP versions 0.5.2, 0.5.4 and 0.5.5. Before building the library you need to download ZFP source code archive from any of the following sites:
-
ZFP 0.5.2 https://computation.llnl.gov/projects/floating-point-compression/download/zfp-0.5.2.tar.gz
-
ZFP 0.5.4 https://computation.llnl.gov/projects/floating-point-compression/download/zfp-0.5.4.tar.gz
-
ZFP 0.5.5 https://computation.llnl.gov/projects/floating-point-compression/download/zfp-0.5.5.tar.gz
Patching and Build on Linux* and macOS*
Download zfp-0.5.x.tar.gz
(x is a neccesary version’s minor number) file from LLNL site to working directory.
Un-archive and patch the surce code using the following commands (it is assumed that zfp-0.5.x.patch
file is in the working directory):
$ tar xf zfp-0.5.x.tar.gz
$ cd zfp-0.5.x
$ patch -p1 < ../zfp-0.5.x.patch.bin
patching file CMakeLists.txt
patching file Config
patching file examples/Makefile
patching file include/bitstream.h
patching file src/CMakeLists.txt
patching file src/inline/bitstream.c
patching file src/Makefile
patching file src/template/compress.c
patching file src/template/decompress.c
patching file tests/Makefile
patching file utils/Makefile
Build the library with standard make
utility
$ make WITH_IPP=yes [ ARCH=ia32|intel64 ]
The option "WITH_IPP=yes
" sets "-DWITH_IPP -I$IPPROOT/include
" options in compiler command line. The option "ARCH=ia32|intel64
"
defines set of Intel® IPP libraries (32- or 64-bits) to be used in build of executables. By default, 64-bit libraries are used.
The above command assumes that IPPROOT
environment variable points to the head of Intel® IPP location.
Note
|
The conditional compilation statements, used for ZFP source code files modification, allow to build original
version of ZFP from the same source code files. For that, you need to build ZFP library using standard "make" command without Intel® IPP-related options.
If WITH_IPP definition doesn’t exist in the compiler options, the ZFP source code files are compiled as unpatched. |
The option "ZFP_WITH_OPENMP=1
" enable support of threading mode via OpenMP for ZFP 0.5.4 and ZFP 0.5.5 versions.
Note
|
Threading with with OpenMP mode is’t supported in MacOSX. |
Downloading, Patching and Build on Windows*
To download ZFP source code files for Windows*, you can use the same URL addresses as for Linux*. However, the patching and building processes on Windows* are different.
Unpacking ZFP Archive
To get the ZFP source code files on Windows* computer, you need to:
-
Download the .gz file from ZFP site
-
Unpack the archive using any of available unzip tools for Windows*. For example, you can use
7-zip
from http://www.7-zip.org/.
Patching Original ZFP Files
There are no standard system tools for patching on Windows*, so you need to use 3-rd party tools. For example, http://gnuwin32.sourceforge.net/packages/patch.htm from Free Software Foundation.
With this tool you need to use the following command line options:
> patch.exe -p1 < <patch file location>
Building ZFP With Intel® IPP on Windows*
The ZFP main directory contains CMake script, which in turn is patched with Intel® IPP definitions. You may use this method as follows:
> mkdir build
> cd build
> cmake -A x64 -DWITH_IPP=yes ..
> msbuild ZFP.sln
The above is the example of preparing working space for of Microsoft* Visual Studio* IDE which was installed on the machine.
To build ZFP library for 32-bit architecture the following CMake command line is required:
> cmake -DWITH_IPP=yes -DARCH=ia32 ..
Technical Support
If you did not register your Intel® software product during installation, please do so now at the Intel® Software Development Products Registration Center. Registration entitles you to free technical support, product updates and upgrades for the duration of the support term.
For general information about Intel technical support, product updates, user forums, FAQs, tips and tricks, and other support questions, please visit (http://www.intel.com/software/products/support).
Note
|
If your distributor provides technical support for this product, please contact them rather than Intel. |
For technical information about the Intel® IPP library, including FAQ’s, tips and tricks, and other support information, please visit the Intel® IPP forum: (http://software.intel.com/en-us/forums/intel-integrated-performance-primitives) and browse the Intel® IPP support page: https://software.intel.com/en-us/intel-ipp-support/.