Geometric Deep Learning: Going beyond Euclidean data

Many scientific fields study data with an underlying structure that is non-Euclidean. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions) and are natural targets for machine-learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural-language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure and in cases where the invariances of these structures are built into networks used to model them.

[1]  Dimitri Van De Ville,et al.  The dynamic functional connectome: State-of-the-art and perspectives , 2017, NeuroImage.

[2]  Alexander M. Bronstein,et al.  Deep Functional Maps: Structured Prediction for Dense Shape Correspondence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  M. Bronstein,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[4]  Ben Glocker,et al.  Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks , 2017, MICCAI.

[5]  Leonidas J. Guibas,et al.  SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[7]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Joshua B. Tenenbaum,et al.  A Compositional Object-Based Approach to Learning Physical Dynamics , 2016, ICLR.

[9]  Joan Bruna,et al.  Topology and Geometry of Half-Rectified Network Optimization , 2016, ICLR.

[10]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[11]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[12]  Alexander M. Bronstein,et al.  Recent Trends, Applications, and Perspectives in 3D Shape Similarity Assessment , 2016, Comput. Graph. Forum.

[13]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[14]  Rob Fergus,et al.  Learning Multiagent Communication with Backpropagation , 2016, NIPS.

[15]  Kenji Kawaguchi,et al.  Deep Learning without Poor Local Minima , 2016, NIPS.

[16]  Jonathan Masci,et al.  Learning shape correspondence with anisotropic convolutional neural networks , 2016, NIPS.

[17]  Daniel Cremers,et al.  Anisotropic Diffusion Descriptors , 2016, Comput. Graph. Forum.

[18]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Stanley Osher,et al.  A Harmonic Extension Approach for Collaborative Ranking , 2016, ArXiv.

[20]  Stéphane Mallat,et al.  Understanding deep convolutional networks , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Pradeep Ravikumar,et al.  Collaborative Filtering with Graph Information: Consistency and Scalable Methods , 2015, NIPS.

[23]  Davide Eynard,et al.  Multimodal Manifold Analysis by Simultaneous Diagonalization of Laplacians , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Qi-Xing Huang,et al.  Dense Human Body Correspondences Using Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Tianqi Chen,et al.  Net2Net: Accelerating Learning via Knowledge Transfer , 2015, ICLR.

[26]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[27]  Ohad Shamir,et al.  On the Quality of the Initial Basin in Overspecified Neural Networks , 2015, ICML.

[28]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[29]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[30]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[31]  Xu Chen,et al.  Deep Haar Scattering Networks , 2015, ArXiv.

[32]  Pierre Vandergheynst,et al.  Learning class‐specific descriptors for deformable shapes using localized spectral convolutional networks , 2015, SGP '15.

[33]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[34]  Max Jaderberg,et al.  Spatial Transformer Networks , 2015, NIPS.

[35]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Davide Eynard,et al.  Shape‐from‐Operator: Recovering Shapes from Intrinsic Operators , 2015, Comput. Graph. Forum.

[37]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  Mark Tygert,et al.  A Mathematical Motivation for Complex-Valued Convolutional Networks , 2015, Neural Computation.

[39]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[40]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[41]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[42]  Yann LeCun,et al.  The Loss Surfaces of Multilayer Networks , 2014, AISTATS.

[43]  Thomas Brox,et al.  Learning to Generate Chairs, Tables and Cars with Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[45]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[46]  Daniel Cremers,et al.  Anisotropic Laplace-Beltrami Operators for Shape Analysis , 2014, ECCV Workshops.

[47]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[48]  Xavier Bresson,et al.  Matrix Completion on Graphs , 2014, NIPS 2014.

[49]  Daniel Cremers,et al.  Dense Non-rigid Shape Correspondence Using Random Forests , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[52]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[54]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[55]  Leonidas J. Guibas,et al.  Wavelets on Graphs via Deep Learning , 2013, NIPS.

[56]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[57]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Pierre Vandergheynst,et al.  Vertex-Frequency Analysis on Graphs , 2013, ArXiv.

[59]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[60]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[61]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[62]  Alexander M. Bronstein,et al.  Coupled quasi‐harmonic bases , 2012, Comput. Graph. Forum.

[63]  Stéphane Mallat,et al.  Invariant Scattering Convolution Networks , 2012, IEEE transactions on pattern analysis and machine intelligence.

[64]  Andrew Y. Ng,et al.  Selecting Receptive Fields in Deep Networks , 2011, NIPS.

[65]  Lukás Burget,et al.  Strategies for training large scale neural network language models , 2011, 2011 IEEE Workshop on Automatic Speech Recognition & Understanding.

[66]  Jürgen Schmidhuber,et al.  A committee of neural networks for traffic sign classification , 2011, The 2011 International Joint Conference on Neural Networks.

[67]  Jure Leskovec,et al.  Friendship and mobility: user movement in location-based social networks , 2011, KDD.

[68]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, ACM Trans. Graph..

[69]  Chao Liu,et al.  Recommender systems with social regularization , 2011, WSDM '11.

[70]  Stéphane Mallat,et al.  Group Invariant Scattering , 2011, ArXiv.

[71]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[72]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[74]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[75]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[76]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[77]  David Rogers,et al.  Extended-Connectivity Fingerprints , 2010, J. Chem. Inf. Model..

[78]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[79]  M. Ovsjanikov,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[80]  Sanjoy Dasgupta,et al.  Which Spatial Partition Trees are Adaptive to Intrinsic Dimension? , 2009, UAI.

[81]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[82]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[83]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[84]  Nicolas Le Roux,et al.  Learning the 2-D Topology of Images , 2007, NIPS.

[85]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[86]  Natasa Przulj,et al.  Biological network comparison using graphlet degree distribution , 2007, Bioinform..

[87]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[88]  Ron Kimmel,et al.  Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[89]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[90]  Ronald R. Coifman,et al.  Diffusion-driven multiscale analysis on manifolds and graphs: top-down and bottom-up constructions , 2005, SPIE Optics + Photonics.

[91]  Richard G. Baraniuk,et al.  The multiscale structure of non-differentiable image manifolds , 2005, SPIE Optics + Photonics.

[92]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[93]  S. Shen-Orr,et al.  Network motifs: simple building blocks of complex networks. , 2002, Science.

[94]  L. Hood,et al.  A Genomic Regulatory Network for Development , 2002, Science.

[95]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[96]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[97]  S. Rosenberg The Laplacian on a Riemannian Manifold: An Introduction to Analysis on Manifolds , 1997 .

[98]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[99]  David J. Field,et al.  What The Statistics Of Natural Images Tell Us About Visual Coding , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[100]  H. L. Morgan The Generation of a Unique Machine Description for Chemical Structures-A Technique Developed at Chemical Abstracts Service. , 1965 .

[101]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[102]  A. Bronstein,et al.  Learning Spectral Descriptors for Deformable Shape Correspondence , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[103]  Tara N. Sainath,et al.  The shared views of four research groups ) , 2012 .

[104]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[105]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[106]  A. Pentland,et al.  Life in the network: The coming age of computational social science: Science , 2009 .

[107]  M. Wardetzky Convergence of the Cotangent Formula: An Overview , 2008 .

[108]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[109]  Eitan Grinspun,et al.  Discrete laplace operators: no free lunch , 2007, Symposium on Geometry Processing.

[110]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[111]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[112]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[113]  S. Mallat A wavelet tour of signal processing , 1998 .

[114]  Ulrich Pinkall,et al.  Computing Discrete Minimal Surfaces and Their Conjugates , 1993, Exp. Math..

[115]  J. Nash The imbedding problem for Riemannian manifolds , 1956 .