Galactos: Computing the Anisotropic 3-Point Correlation Function for 2 Billion Galaxies

The nature of dark energy and the complete theory of gravity are two central questions currently facing cosmology. A vital tool for addressing them is the 3-point correlation function (3PCF), which probes deviations from a spatially random distribution of galaxies. However, the 3PCF's formidable computational expense has prevented its application to astronomical surveys comprising millions to billions of galaxies. We present Galactos, a high-performance implementation of a novel, O(N2) algorithm that uses a load-balanced k-d tree and spherical harmonic expansions to compute the anisotropic 3PCF. Our implementation is optimized for the Intel Xeon Phi architecture, exploiting SIMD parallelism, instruction and thread concurrency, and significant L1 and L2 cache reuse, reaching 39% of peak performance on a single node. Galactos scales to the full Cori system, achieving 9.8 PF (peak) and 5.06 PF (sustained) across 9636 nodes, making the 3PCF easily computable for all galaxies in the observable universe.

[1]  W. M. Wood-Vasey,et al.  The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample , 2016, 1607.03155.

[2]  Cornelius Rampf,et al.  Lagrangian perturbations and the matter bispectrum II: the resummed one-loop correction to the matter bispectrum , 2012, 1203.4261.

[3]  L. Wasserman,et al.  Fast Algorithms and Efficient Statistics: N-Point Correlation Functions , 2000, astro-ph/0012333.

[4]  Max Tegmark,et al.  Cosmic complementarity: H(0) and Omega(m) from combining CMB experiments and redshift surveys , 1998 .

[5]  C. Martin.  2015 , 2015, Les 25 ans de l’OMC: Une rétrospective en photos.

[6]  Patrick McDonald,et al.  Clustering of dark matter tracers: generalizing bias for the coming era of precision LSS , 2009, 0902.0991.

[7]  Hal Finkel,et al.  HACC: Simulating Sky Surveys on State-of-the-Art Supercomputing Architectures , 2014, 1410.2805.

[8]  A. Hamilton,et al.  Linear redshift distortions: A Review , 1997, astro-ph/9708102.

[9]  G. Kowal,et al.  DENSITY STUDIES OF MHD INTERSTELLAR TURBULENCE: STATISTICAL MOMENTS, CORRELATIONS AND BISPECTRUM , 2008, 0811.0822.

[10]  Adam G. Riess,et al.  Observational probes of cosmic acceleration , 2012, 1201.2434.

[11]  Edward J. Wollack,et al.  SEVEN-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE (WMAP) OBSERVATIONS: POWER SPECTRA AND WMAP-DERIVED PARAMETERS , 2010, 1001.4635.

[12]  J.N.Fry,et al.  Redshift distortions of galaxy correlation functions , 1993 .

[13]  Aidan J. Connolly,et al.  A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid , 2007 .

[14]  Deborah Bard,et al.  Cosmological calculations on the GPU , 2012, Astron. Comput..

[15]  A. Kashlinsky,et al.  Large-scale structure in the Universe , 1991, Nature.

[16]  Eric V. Linder,et al.  Cosmic growth history and expansion history , 2005 .

[17]  Surendra Byna,et al.  BD-CATS: big data clustering at trillion particle scale , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[18]  Joshua A. Frieman,et al.  The Bispectrum as a Signature of Gravitational Instability in Redshift Space , 1998, astro-ph/9808305.

[19]  Hal Finkel,et al.  HACC: extreme scaling and performance across diverse architectures , 2016 .

[20]  C. Frye,et al.  Spherical Harmonics in p Dimensions , 2012, 1205.3548.

[21]  A. James.  2010 , 2011, Philo of Alexandria: an Annotated Bibliography 2007-2016.

[22]  D. Eisenstein,et al.  Computing the Three-Point Correlation Function of Galaxies in $\mathcal{O}(N^2)$ Time , 2015, 1506.02040.

[23]  Zachary Slepian,et al.  A practical computational method for the anisotropic redshift-space three-point correlation function , 2017, 1709.10150.

[24]  Shinji Tsujikawa,et al.  Dynamics of dark energy , 2006 .

[25]  Francisco-Shu Kitaura,et al.  The clustering of galaxies in the SDSS-III Baryon Oscillation Spectroscopic Survey: RSD measurement from the power spectrum and bispectrum of the DR12 BOSS galaxies , 2016, 1606.00439.

[26]  A. J. Connolly,et al.  The Effect of Large-Scale Structure on the SDSS Galaxy Three-Point Correlation Function , 2006, astro-ph/0602548.

[27]  Florence March,et al.  2016 , 2016, Affair of the Heart.

[28]  Christopher Hirata,et al.  Relative velocity of dark matter and baryonic fluids and the formation of the first structures , 2010, 1005.2416.

[29]  I. Sevilla,et al.  Application of GPUs for the Calculation of Two Point Correlation Functions in Cosmology , 2012, 1204.6630.

[30]  Zachary Slepian,et al.  Modelling the large-scale redshift-space 3-point correlation function of galaxies , 2016, 1607.03109.

[32]  Pramodita Sharma.  2012 , 2013, Les 25 ans de l’OMC: Une rétrospective en photos.

[33]  Andrew W. Moore,et al.  Multi-Tree Methods for Statistics on Very Large Datasets in Astronomy , 2004 .

[34]  John Shalf,et al.  Billion-particle SIMD-friendly two-point correlation on large-scale HPC cluster systems , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[35]  G. Bernstein,et al.  The skewness of the aperture mass statistic , 2003 .

[36]  Davis,et al.  THREE-POINT CORRELATION FUNCTIONS OF SDSS GALAXIES: CONSTRAINING GALAXY-MASS BIAS , 2010, 1012.3462.

[37]  Sebastian Pueblas,et al.  Cosmology and the Bispectrum , 2006 .

[38]  Ashley J. Ross,et al.  Detection of baryon acoustic oscillation features in the large-scale three-point correlation function of SDSS BOSS DR12 CMASS galaxies , 2016, 1607.06097.

[39]  William B. March.  Multi-tree algorithms for computational statistics and phyiscs , 2013 .

[40]  Donald P. Schneider,et al.  The power spectrum and bispectrum of SDSS DR11 BOSS galaxies – I. Bias and gravity , 2014, 1407.5668.