Portably Improving Uintah ’ s Readiness for Exascale Systems Through the Use of Kokkos

Uncertainty and diversity in future HPC systems, including those for exascale, makes portable codebases desirable. To ease future ports, the Uintah Computational Framework has adopted the Kokkos C++ Performance Portability Library. This paper describes infrastructure advancements and performance improvements using partitioning functionality recently added to Kokkos within Uintah’s MPI+Kokkos hybrid parallelism approach. Results are presented for two challenging calculations that have been refactored to support Kokkos::OpenMP and Kokkos::Cuda back-ends. These results demonstrate performance improvements up to (i) 2.66x when refactoring for portability, (ii) 81.59x when adding loop-level parallelism via Kokkos back-ends, and (iii) 2.63x when more efficiently using a node. Good strong-scaling characteristics to 442,368 threads across 1728 Knights Landing processors are also shown. These improvements have been achieved with little added overhead (sub-millisecond, consuming up to 0.18% of per-timestep time). Kokkos adoption and refactoring lessons are also discussed. Portably Improving Uintah’s Readiness for Exascale Systems Through the Use of Kokkos John K. Holmen, Brad Peterson, Alan Humphrey, Daniel Sunderland, Oscar H. Dı́az-Ibarra, Jeremy N. Thornock, Martin Berzins aScientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT 84112 bSandia National Laboratories, PO Box 5800 / MS 1418, Albuquerque, NM 87175 cInstitute for Clean and Secure Energy, University of Utah, Salt Lake City, UT 84112