DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications

Convolutional Neural Networks (CNNs) are a cornerstone of the Deep Learning toolbox and have led to many breakthroughs in Artificial Intelligence. These networks have mostly been developed for regular Euclidean domains such as those supporting images, audio, or video. Because of their success, CNN-based methods are becoming increasingly popular in Cosmology. Cosmological data often comes as spherical maps, which make the use of the traditional CNNs more complicated. The commonly used pixelization scheme for spherical maps is the Hierarchical Equal Area isoLatitude Pixelisation (HEALPix). We present a spherical CNN for analysis of full and partial HEALPix maps, which we call DeepSphere. The spherical CNN is constructed by representing the sphere as a graph. Graphs are versatile data structures that can act as a discrete representation of a continuous manifold. Using the graph-based representation, we define many of the standard CNN operations, such as convolution and pooling. With filters restricted to being radial, our convolutions are equivariant to rotation on the sphere, and DeepSphere can be made invariant or equivariant to rotation. This way, DeepSphere is a special case of a graph CNN, tailored to the HEALPix sampling of the sphere. This approach is computationally more efficient than using spherical harmonics to perform convolutions. We demonstrate the method on a classification problem of weak lensing mass maps from two cosmological models and compare the performance of the CNN with that of two baseline classifiers. The results show that the performance of DeepSphere is always superior or equal to both of these baselines. For high noise levels and for data covering only a smaller fraction of the sphere, DeepSphere achieves typically 10% better classification accuracy than those baselines. Finally, we show how learned filters can be visualized to introspect the neural network.

[1]  Andreas Geiger,et al.  SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images , 2018, ECCV.

[2]  J. Berger,et al.  Detecting cosmic strings in the CMB with the Canny algorithm , 2007, 0709.0982.

[3]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[4]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  C. A. Oxborrow,et al.  Planck2015 results , 2015, Astronomy & Astrophysics.

[6]  R. Nichol,et al.  Dark energy survey year 1 results: curved-sky weak lensing mass map , 2017, 1708.01535.

[7]  Wouter Boomsma,et al.  Spherical convolutions and their application in molecular modelling , 2017, NIPS.

[8]  J. Dunkley,et al.  Recent discoveries from the cosmic microwave background: a review of recent progress , 2017, Reports on progress in physics. Physical Society.

[9]  Michelle Lochner,et al.  Machine learning cosmological structure formation , 2018, Monthly Notices of the Royal Astronomical Society.

[10]  Nicolas Tremblay,et al.  Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[11]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[12]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[13]  Yinda Zhang,et al.  PanoContext: A Whole-Room 3D Context Model for Panoramic Scene Understanding , 2014, ECCV.

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[16]  Martin J. Mohlenkamp A fast transform for spherical harmonics , 1997 .

[17]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[18]  Pierre Vandergheynst,et al.  Stationary Signal Processing on Graphs , 2016, IEEE Transactions on Signal Processing.

[19]  Barnabás Póczos,et al.  Estimating Cosmological Parameters from the Dark Matter Distribution , 2016, ICML.

[20]  Kristen Grauman,et al.  Flat2Sphere: Learning Spherical Convolution for Fast Features from 360° Imagery , 2017, NIPS 2017.

[21]  S. Kohn,et al.  Reionization Models Classifier using 21cm Map Deep Learning , 2017, Proceedings of the International Astronomical Union.

[22]  D. A. García-Hernández,et al.  University of Birmingham The Fourteenth Data Release of the Sloan Digital Sky Survey: , 2017 .

[23]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[24]  Ben Glocker,et al.  Metric learning with spectral graph convolutions on brain connectivity networks , 2018, NeuroImage.

[25]  David Alonso,et al.  Cosmology with a SKA HI intensity mapping survey , 2015, 1501.03989.

[26]  K. Gorski,et al.  HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on the Sphere , 2004, astro-ph/0409513.

[27]  P. Paolucci,et al.  The “Cubed Sphere” , 1996 .

[28]  Sanja Fidler,et al.  3D Graph Neural Networks for RGBD Semantic Segmentation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[30]  B. Winkel,et al.  HI4PI: a full-sky H i survey based on EBHIS and GASS , 2016, 1610.06175.

[31]  C. A. Oxborrow,et al.  Planck 2013 results. I. Overview of products and scientific results , 2013, 1502.01582.

[32]  Adam Amara,et al.  Fast generation of covariance matrices for weak lensing , 2018, Journal of Cosmology and Astroparticle Physics.

[33]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[34]  W. M. Wood-Vasey,et al.  The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample , 2016, 1607.03155.

[35]  Daniel J. Hsu,et al.  Non-Gaussian information from weak lensing data via deep learning , 2018, ArXiv.

[36]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[37]  Ben Glocker,et al.  Spectral Graph Convolutions for Population-based Disease Prediction , 2017, MICCAI.

[38]  Mark Tygert,et al.  Fast Algorithms for Spherical Harmonic Expansions , 2006, SIAM J. Sci. Comput..

[39]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[40]  Xavier Bresson,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[41]  Pascal Fua,et al.  Geodesic Convolutional Shape Optimization , 2018, ICML.

[42]  Patrick Hop,et al.  Geometric Deep Learning Autonomously Learns Chemical Features That Outperform Those Engineered by Domain Experts. , 2018, Molecular pharmaceutics.

[43]  Siamak Ravanbakhsh,et al.  Analysis of Cosmic Microwave Background with Deep Learning , 2018, ICLR.

[44]  Peter Schneider,et al.  Weak Gravitational Lensing , 2005, astro-ph/0509252.

[45]  Krista A. Ehinger,et al.  Recognizing scene viewpoint using panoramic place representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[47]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[48]  Jonathan Masci,et al.  Learning shape correspondence with anisotropic convolutional neural networks , 2016, NIPS.

[49]  M. Tomasi,et al.  Convolutional neural networks on the HEALPix sphere: a pixel-based algorithm and its application to CMB data analysis , 2019, Astronomy & Astrophysics.

[50]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[51]  Harmonic inpainting of the cosmic microwave background sky: Formulation and error estimate , 2008, 0804.0527.

[52]  Mikhail Belkin,et al.  Convergence of Laplacian Eigenmaps , 2006, NIPS.

[53]  Dag Sverre Seljebotn,et al.  Libsharp – spherical harmonic transforms revisited , 2013, 1303.4945.

[54]  Edward J. Wollack,et al.  FIVE-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE OBSERVATIONS: COSMOLOGICAL INTERPRETATION , 2008, 0803.0547.

[55]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[56]  P. Schneider,et al.  KiDS-450: cosmological parameter constraints from tomographic weak gravitational lensing , 2016, 1606.05338.

[57]  O. Wucknitz Gravitational Lensing , 2007, Large-Scale Peculiar Motions.

[58]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[59]  Pascal Frossard,et al.  Graph-Based Classification of Omnidirectional Images , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[60]  Kenneth Patton,et al.  Cosmological constraints from the convergence 1-point probability distribution , 2016, Monthly Notices of the Royal Astronomical Society.

[61]  Cullan Howlett,et al.  L-PICOLA: A parallel code for fast dark matter simulation , 2015, Astron. Comput..

[62]  M. Bartelmann Gravitational lensing , 2010, 1010.3829.

[63]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.