Deep learning insights into cosmological structure formation

While the evolution of linear initial conditions present in the early universe into extended halos of dark matter at late times can be computed using cosmological simulations, a theoretical understanding of this complex process remains elusive. Here, we build a deep learning framework to learn this non-linear relationship, and develop techniques to physically interpret the learnt mapping. A three-dimensional convolutional neural network (CNN) is trained to predict the mass of dark matter halos from the initial conditions. We find no change in the predictive accuracy of the model if we retrain the model removing anisotropic information from the inputs. This suggests that the features learnt by the CNN are equivalent to spherical averages over the initial conditions. Our results indicate that interpretable deep learning frameworks can provide a powerful tool for extracting insight into cosmological structure formation.

[1]  William H. Press,et al.  Formation of Galaxies and Clusters of Galaxies by Self-Similar Gravitational Condensation , 1974 .

[2]  M. Rees,et al.  Core condensation in heavy halos: a two-stage theory for galaxy formation and clustering , 1978 .

[3]  Misha Denil,et al.  Predicting Parameters in Deep Learning , 2014 .

[4]  M. Rees,et al.  Formation of galaxies and large-scale structure with cold dark matter , 1985, Nature.

[5]  T. Naab,et al.  GalaxyNet: connecting galaxies and dark matter haloes with deep neural networks and reinforcement learning in large volumes , 2020, Monthly Notices of the Royal Astronomical Society.

[6]  V. Springel The Cosmological simulation code GADGET-2 , 2005, astro-ph/0505010.

[7]  J. Bond,et al.  The Peak-Patch Picture of Cosmic Catalogs. II. Validation , 1996 .

[8]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Deborah Silver,et al.  Feature Visualization , 1994, Scientific Visualization.

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  Lehman H. Garrison,et al.  A Hybrid Deep Learning Approach to Cosmological Constraints from Galaxy Redshift Surveys , 2019 .

[12]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Neural physical engines for inferring the halo mass distribution function , 2020, 1909.06379.

[14]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[15]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[16]  Danilo Comminiello,et al.  Group sparse regularization for deep neural networks , 2016, Neurocomputing.

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  Alun D. Preece,et al.  Interpretability of deep learning models: A survey of results , 2017, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[19]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  H. Peiris,et al.  An interpretable machine-learning framework for dark matter halo formation , 2019, Monthly Notices of the Royal Astronomical Society.

[21]  A. Pontzen,et al.  pynbody: N-Body/SPH analysis for python , 2013 .

[22]  S. Ho,et al.  Neural Networks as Optimal Estimators to Marginalize Over Baryonic Effects , 2020, The Astrophysical Journal.

[23]  Chandan Singh,et al.  Definitions, methods, and applications in interpretable machine learning , 2019, Proceedings of the National Academy of Sciences.

[24]  Ravi K. Sheth Giuseppe Tormen Large scale bias and the peak background split , 1999 .

[25]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[27]  A. Klypin,et al.  DARK MATTER HALOS IN THE STANDARD COSMOLOGICAL MODEL: RESULTS FROM THE BOLSHOI SIMULATION , 2010, 1002.3660.

[28]  Renato Renner,et al.  Operationally meaningful representations of physical systems in neural networks , 2020, Mach. Learn. Sci. Technol..

[29]  M. Halpern,et al.  FIVE-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE OBSERVATIONS: LIKELIHOODS AND PARAMETERS FROM THE WMAP DATA , 2008, 0803.0586.

[30]  G. Efstathiou,et al.  Numerical techniques for large cosmological N-body simulations , 1985 .

[31]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[32]  Michelle Lochner,et al.  Machine learning cosmological structure formation , 2018, Monthly Notices of the Royal Astronomical Society.

[33]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[34]  H. M. P. Couchman,et al.  The mass function of dark matter haloes , 2000, astro-ph/0005260.

[35]  Barnabás Póczos,et al.  Estimating Cosmological Parameters from the Dark Matter Distribution , 2016, ICML.

[36]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Philippe Berger,et al.  A volumetric deep Convolutional Neural Network for simulation of dark matter halo catalogues , 2018, Monthly Notices of the Royal Astronomical Society.

[38]  Hiranya Peiris,et al.  GenetIC—A New Initial Conditions Generator to Support Genetically Modified Zoom Simulations , 2020, The Astrophysical Journal Supplement Series.

[39]  J. Bond,et al.  The Peak-Patch Picture of Cosmic Catalogs. I. Algorithms , 1996 .

[40]  Prabhat,et al.  CosmoFlow: Using Deep Learning to Learn the Universe at Scale , 2018, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis.

[41]  Volker Springel,et al.  Resolving cosmic structure formation with the Millennium-II simulation , 2009, 0903.3041.

[42]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[43]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[44]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[45]  Tom Charnock,et al.  Painting halos from cosmic density fields of dark matter with physically motivated neural networks , 2019, Physical Review D.

[46]  R. Ellis,et al.  The 2dF Galaxy Redshift Survey: the power spectrum and the matter content of the Universe , 2001, astro-ph/0105252.

[47]  Arvind Satyanarayan,et al.  The Building Blocks of Interpretability , 2018 .

[48]  S. White,et al.  A Universal Density Profile from Hierarchical Clustering , 1996, astro-ph/9611107.

[49]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[50]  V. Springel,et al.  GADGET: a code for collisionless and gasdynamical cosmological simulations , 2000, astro-ph/0003162.

[51]  Shy Genel,et al.  From Dark Matter to Galaxies with Convolutional Neural Networks , 2019, ArXiv.

[52]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[53]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[54]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[55]  R. Sheth,et al.  An excursion set model of hierarchical clustering: ellipsoidal collapse and the moving barrier , 2001, astro-ph/0105113.

[56]  Wei Zhang,et al.  From Dark Matter to Galaxies with Convolutional Networks , 2019, ArXiv.

[58]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[59]  Ofer Lahav,et al.  The Cosmological Parameters , 2003 .

[60]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[61]  R. Sheth,et al.  Ellipsoidal collapse and an improved model for the number and spatial distribution of dark matter haloes , 1999, astro-ph/9907024.

[62]  J. R. Bond,et al.  Excursion set mass functions for hierarchical Gaussian fluctuations , 1991 .

[63]  Yu Feng,et al.  Cosmological reconstruction from galaxy light: neural network based light-matter connection , 2018, Journal of Cosmology and Astroparticle Physics.

[64]  Wei Chen,et al.  Learning to predict the cosmological structure formation , 2018, Proceedings of the National Academy of Sciences.

[65]  Huaiyu Zhu On Information and Sufficiency , 1997 .