HyperGCN: Hypergraph Convolutional Networks for Semi-Supervised Classification

Graph-based semi-supervised learning (SSL) is an important learning problem where the goal is to assign labels to initially unlabeled nodes in a graph. Graph Convolutional Networks (GCNs) have recently been shown to be effective for graph-based SSL problems. GCNs inherently assume existence of pairwise relationships in the graph-structured data. However, in many real-world problems, relationships go beyond pairwise connections and hence are more complex. Hypergraphs provide a natural modeling tool to capture such complex relationships. In this work, we explore the use of GCNs for hypergraph-based SSL. In particular, we propose HyperGCN, an SSL method which uses a layer-wise propagation rule for convolutional neural networks operating directly on hypergraphs. To the best of our knowledge, this is the first principled adaptation of GCNs to hypergraphs. HyperGCN is able to encode both the hypergraph structure and hypernode features in an effective manner. Through detailed experimentation, we demonstrate HyperGCN's effectiveness at hypergraph-based SSL.

[1]  Austin R. Benson,et al.  Planted Hitting Set Recovery in Hypergraphs , 2019, ArXiv.

[2]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[3]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[4]  Pan Li,et al.  HS2: Active learning over hypergraphs with pointwise and pairwise queries , 2019, AISTATS.

[5]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[6]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[7]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[8]  Anand Louis,et al.  Hypergraph Markov Operators, Eigenvalues and Approximation Algorithms , 2014, STOC.

[9]  Balaraman Ravindran,et al.  Extended Discriminative Random Walk: A Hypergraph Approach to Multi-View Multi-Relational Transductive Learning , 2015, IJCAI.

[10]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[11]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[12]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[13]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[14]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[15]  Partha Pratim Talukdar,et al.  Graph-Based Semi-Supervised Learning , 2014, Graph-Based Semi-Supervised Learning.

[16]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[17]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[18]  Bernhard Schölkopf,et al.  Cluster Kernels for Semi-Supervised Learning , 2002, NIPS.

[19]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[20]  Qiang Ma,et al.  Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification , 2018, WWW.

[21]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[22]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[23]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[24]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yiqun Liu,et al.  Learning on Partial-Order Hypergraphs , 2018, WWW.

[26]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[27]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[28]  S. Mallat A wavelet tour of signal processing , 1998 .

[29]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[30]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[31]  Matthias Hein,et al.  The Total Variation on Hypergraphs - Learning on Hypergraphs Revisited , 2013, NIPS.

[32]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.

[33]  Olgica Milenkovic,et al.  Revisiting Decomposable Submodular Function Minimization with Incidence Relations , 2018, NeurIPS.

[34]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[36]  Sarah Parisot,et al.  Learning Conditioned Graph Structures for Interpretable Visual Question Answering , 2018, NeurIPS.

[37]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[38]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[39]  Chenzi Zhang,et al.  Spectral Properties of Hypergraph Laplacian and Approximation Algorithms , 2016, J. ACM.

[40]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[41]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[42]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[43]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[44]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[45]  Serge J. Belongie,et al.  Higher order learning with graphs , 2006, ICML.

[46]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[47]  Chenzi Zhang,et al.  Re-revisiting Learning on Hypergraphs: Confidence Interval and Subgradient Method , 2017, ICML.

[48]  Abhinav Gupta,et al.  Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Marcello Pelillo,et al.  A Game-Theoretic Approach to Hypergraph Clustering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  George Michailidis,et al.  Graph-Based Semisupervised Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Yu Gong,et al.  Exact-K Recommendation via Maximal Clique Optimization , 2019, KDD.

[52]  My T. Thai,et al.  Approximate k-Cover in Hypergraphs: Efficient Algorithms, and Applications , 2019, ArXiv.

[53]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[54]  Luís C. Lamb,et al.  Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[55]  Olgica Milenkovic,et al.  Submodular Hypergraphs: p-Laplacians, Cheeger Inequalities and Spectral Clustering , 2018, ICML.

[56]  Stephan Günnemann,et al.  Dual-Primal Graph Convolutional Networks , 2018, ArXiv.

[57]  T.-H. Hubert Chan,et al.  Generalizing the Hypergraph Laplacian via a Diffusion Process with Mediators , 2018, COCOON.

[58]  Yue Gao,et al.  Hypergraph Neural Networks , 2018, AAAI.

[59]  Michael Dinitz,et al.  The Densest k-Subhypergraph Problem , 2016, APPROX-RANDOM.

[60]  Prateek Yadav,et al.  Confidence-based Graph Convolutional Networks for Semi-Supervised Learning , 2019, AISTATS.

[61]  Jason Weston,et al.  Deep learning via semi-supervised embedding , 2008, ICML '08.