SWIFT: Task-based Parallelism, Asynchronous Communication, and Vectorization for HPC Performance

We present a new open-source cosmological code, called swift, designed to solve the equations of hydrodynamics using a particle-based approach (Smooth Particle Hydrodynamics) on hybrid shared / distributed-memory architectures. Swift was designed from the bottom up to provide excellent strong scaling on both commodity clusters (Tier-2 systems) and Top100-supercomputers (Tier-0 systems), without relying on architecture-specific features or specialized accelerator hardware. This performance is due to three main computational approaches: - Task-based parallelism for shared-memory parallelism, which provides fine-grained load balancing and thus strong scaling on large numbers of cores. - Graph-based and genetic algorithm-based domain decomposition, which uses the task graph to decompose the simulation domain such that the work, as opposed to just the data, as is the case with most partitioning schemes, is equally distributed across all nodes. - Fully dynamic and asynchronous communication, in which communication is modeled as just another task in the task-based scheme, sending data whenever it is ready and deferring on tasks that rely on data from other nodes until it arrives, - Explicit vectorization of the core kernel to exploit all the available FLOPS on architectures such as Intel® Xeon Phi™. In order to use these approaches, the code had to be rewritten from scratch, and the algorithms therein adapted to the task-based paradigm. As a result, we can show upwards of 60% parallel efficiency for moderate-sized problems when increasing the number of cores 512-fold on x86 architecture making SWIFT more than an order of magnitude faster than current alternative software.