Determining the Number of OpenMP* Threads
The OpenMP* run-time library responds to the environment variable OMP_NUM_THREADS. Intel® oneAPI Math Kernel Library (oneMKL) also has other mechanisms to set the number of OpenMP threads, such as theMKL_NUM_THREADS or MKL_DOMAIN_NUM_THREADS environment variables (see Using Additional Threading Control).
Make sure that the relevant environment variables have the same and correct values on all the nodes. Intel® oneAPI Math Kernel Library (oneMKL) does not set the default number of OpenMP threads to one, but depends on the OpenMP libraries used with the compiler to set the default number. For the threading layer based on the Intel compiler (mkl_intel_thread.lib), this value is the number of CPUs according to the OS.
Avoid over-prescribing the number of OpenMP threads, which may occur, for instance, when the number of MPI ranks per node and the number of OpenMP threads per node are both greater than one. The number of MPI ranks per node multiplied by the number of OpenMP threads per node should not exceed the number of hardware threads per node.
The OMP_NUM_THREADS environment variable is assumed in the discussion below.
Set OMP_NUM_THREADS so that the product of its value and the number of MPI ranks per node equals the number of real processors or cores of a node. If the Intel® Hyper-Threading Technology is enabled on the node, use only half number of the processors that are visible on Windows OS.
For Cluster Sparse Solver, set the number of OpenMP threads to a number greater than one because the implementation of the solver only supports a multithreaded algorithm.