Spherical CNNs

Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression.

[1]  D. Rockmore.  Recent progress and applications in group FFTs , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[2]  G. Folland.  A course in abstract harmonic analysis , 1995 .

[3]  Stefan Kunis,et al.  Fast spherical Fourier algorithms , 2003 .

[4]  Lorenz C. Blum,et al.  970 million druglike small molecules for virtual screening in the chemical universe database GDB-13. , 2009, Journal of the American Chemical Society.

[5]  L. Nachbin,et al.  The Haar integral , 1965 .

[6]  Max Welling,et al.  Steerable CNNs , 2016, ICLR.

[7]  K. Müller,et al.  Fast and accurate modeling of molecular atomization energies with machine learning. , 2011, Physical review letters.

[8]  Gabriele Steidl,et al.  Fast and stable algorithms for discrete spherical Fourier transforms , 1998 .

[9]  Sean S. B. Moore,et al.  FFTs for the 2-Sphere-Improvements and Variations , 1996 .

[10]  Patrick H. Worley,et al.  Algorithm 888: Spherical Harmonic Transform Algorithms , 2008, TOMS.

[11]  T. Chan,et al.  Shape Registration with Spherical Cross Correlation , 2008 .

[12]  Kristen Grauman,et al.  Flat2Sphere: Learning Spherical Convolution for Fast Features from 360° Imagery , 2017, NIPS 2017.

[13]  Andreas Ziehe,et al.  Learning Invariant Representations of Molecules for Atomization Energy Prediction , 2012, NIPS.

[14]  Maurice Weiler,et al.  Learning Steerable Filters for Rotation Equivariant CNNs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Barnabás Póczos,et al.  Deep Learning with Sets and Point Clouds , 2016, ICLR.

[16]  Max Welling,et al.  Convolutional Networks for Spherical Signals , 2017, ArXiv.

[17]  Bo Li,et al.  Large-Scale 3D Shape Retrieval from ShapeNet Core55 , 2016, 3DOR@Eurographics.

[18]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[19]  D. Maslen.  Efficient computation of Fourier transforms on compact groups , 1998 .

[20]  Bernhard Schölkopf,et al.  Local Group Invariant Representations via Orbit Embeddings , 2016, AISTATS.

[21]  Wouter Boomsma,et al.  Spherical convolutions and their application in molecular modelling , 2017, NIPS.

[22]  Michael E. Taylor,et al.  Noncommutative Harmonic Analysis , 1986 .

[23]  Nathaniel Virgo,et al.  Permutation-equivariant neural networks applied to dynamics prediction , 2016, ArXiv.

[24]  D. Healy,et al.  Computing Fourier Transforms and Convolutions on the 2-Sphere , 1994 .

[25]  Stephan J. Garbin,et al.  Harmonic Networks: Deep Translation and Rotation Equivariance , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  D. Rockmore,et al.  FFTs on the Rotation Group , 2008 .

[27]  Pedro M. Domingos,et al.  Deep Symmetry Networks , 2014, NIPS.

[28]  M. Sugiura.  Unitary Representations and Harmonic Analysis , 1990 .

[29]  Daniel Potts,et al.  A fast algorithm for nonequispaced Fourier transforms on the rotation group , 2009, Numerical Algorithms.

[30]  Kostas Daniilidis,et al.  Correspondence-free Structure from Motion , 2007, International Journal of Computer Vision.

[31]  Sander Dieleman,et al.  Rotation-invariant convolutional neural networks for galaxy morphology prediction , 2015, ArXiv.

[32]  Koray Kavukcuoglu,et al.  Exploiting Cyclic Symmetry in Convolutional Neural Networks , 2016, ICML.

[33]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.