Building Redis* with Intel® DPC++/C++ Compiler on Linux*

Published: 11/01/2021

Contributor: Lin, Jie jie.lin@intel.com

Step 1 Overview

From Redis website https://redis.io/, "Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster."

Inter-procedural Optimization (IPO) that is used by Intel® C/C++ Compiler classic is an automatic, multi-step process that a static, topological analysis that span all of your source files are performed by the compiler to determine where you can benefit from specific optimizations. Now, the new Intel® DPC++/C++ Compiler which is LLVM-based uses Link Time Optimization (LTO) technology which is similar to the IPO concept. For more information on LLVM LTO, refer to

https://llvm.org/docs/LinkTimeOptimization.html.

While IPO is a static analysis of your program, Profile-Guided Optimizations (PGO) is a dynamic analysis that provides information to the compiler about areas of an application that are most frequently executed. Using PGO the compiler can be more selective and specific in optimizing the application. Intel® DPC++/C++ Compiler supports PGO by using profiles that can provide execution counts for instructions in the code and information on branches taken and function invocation. For more information, refer to Profile Guided Optimization https://clang.llvm.org/docs/UsersManual.html#id40

This article introduces how to build Redis with Intel® DPC++/C++ Compiler with both oridinary optimization options and advanced optimization options that uses IPO and PGO on Linux* platforms.

Step 2 Version Information

The instructions have been verified with Intel® oneAPI BaseToolkit 2021.4 on Ubuntu* 20.04.3 LTS (Focal Fossa).

GCC version information :

gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.

Step 3 Downloading Redis Source Code

$ git clone https://github.com/redis/redis.git

  1. Prerequisites
    Download and install Intel® oneAPI Base Toolkit
    The download link could be found at https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html
  2. Configuration
    Set the Intel® DPC++/C++ Compiler environment variables by running the following commands assuming the default installation path of oneAPI is used:
    $export PATH=/opt/intel/oneapi/compiler/latest/linux/bin:$PATH
    $export LD_LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/lib:/opt/intel/oneapi/compiler/latest/linux/compiler/lib/intel64_lin:$LD_LIBRARY_PATH
    $export LIBRARY_PATH=/opt/intel/oneapi/compiler/latest/linux/lib:$LIBRARY_PATH

Step 4 – Building Redis with Ordinary Optimization Options

Redis uses Makefile build system. So if you want to use Intel® DPC++/C++ Compiler, you can change the Makefile environment variables to reference driver names icx and icpx in Intel® DPC++/C++ Compiler. Type below command in the root directory of Redis source tree to enable compiler optimizations using AVX512 instruction set :

make CC=icx CXX=icpx CFLAGS=' -O3 -xCORE-AVX512 ' CXXFLAGS=' -O3 -xCORE-AVX512 '

Step 5 – Building Redis with IPO and PGO

In addtion to the ordinary optimization options of Intel® DPC++/C++ Compiler, we could further optimize Redis performance by enabling IPO and PGO.

  1. IPO

    For IPO build, a bunch of LLVM binaries are used such as 'llvm-ar', 'llvm-ranlib' and 'llvm-nm'. Intel® oneAPI BaseToolkit 2021.4 does not provide 'llvm-nm'. So you must download it by yourself from https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20HYPERLINK "https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/clang+llvm-12.0.0-x86_64-linux-gnu-ubuntu-20.04.tar.xz".04.tar.xz and copy the 'llvm-nm' downloaded into your oneAPI installation directory /opt/intel/oneapi/compiler/latest/linux/bin (assuming you are using the default installation path of oneAPI).

    Now for IPO build, you could type below command in the root directory of Redis source tree :

    make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -ipo ' CXXFLAGS=' -O3 -xHost -ipo ' LDFLAGS=' -ipo ' AR='llvm-ar' RANLIB='llvm-ranlib' NM='llvm-nm'
     
  2. PGO

    Redis performance could be better if you also enable PGO for build. The PGO enabling requires below four phases :
    1. The first phase is instrumentation by adding option '-fprofile-generate'.

      make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -fprofile-generate ' CXXFLAGS=' -O3 -xHost -fprofile-generate ' LDFLAGS=' -fprofile-generate '
       
    2. run the instumented binaries seperately to get profiling raw data (*.profraw files) .

      For example, below two commands would get the profiling raw data files for 'redis-server' and 'redis-benchmark' separately.

      LLVM_PROFILE_FILE="server-%m.profraw" ./redis/src/redis-server ./redis/src/redis.conf"
      LLVM_PROFILE_FILE="benchmark-%m.profraw" ./redis/src/redis-benchmark -h 127.0.0.1 -p 6379 -n 10000000 -r 1000000 -c 50 -d 32 -t get,set,incr,lpush,lpop -P 64 -q
       
    3. create profiling data file with input raw *.profraw files. For example,

      "llvm-profdata merge --output=benchmark.profdata benchmark*.profraw"
      "llvm-profdata merge --output=server.profdata server*.profraw"
       
    4. Rebuild with the corresponding profiling data (*.profdata file). For example, below command builds with both IPO and the profiling data input with option '-fprofile-use' :

      make CC=icx CXX=icpx CFLAGS=' -O3 -xHost -fprofile-use=./server.profdata -ipo ' CXXFLAGS=' -O3 -xHost -fprofile-use=./server.profdata -ipo ' LDFLAGS=' -fprofile-use=./server.profdata -ipo ' AR='llvm-ar' RANLIB='llvm-ranlib' NM='llvm-nm'

Step 6 - Running Redis Benchmark

After build, Redis binaries will appear in the "src" sub-directory. Now you could try to run the benchmark following this link : https://redis.io/topics/benchmarks

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.