HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs

In many real-world network datasets such as co-authorship, co-citation, email communication, etc., relationships are complex and go beyond pairwise. Hypergraphs provide a flexible and natural modeling tool to model such complex relationships. The obvious existence of such complex relationships in many real-world networks naturaly motivates the problem of learning with hypergraphs. A popular learning paradigm is hypergraph-based semi-supervised learning (SSL) where the goal is to assign labels to initially unlabeled vertices in a hypergraph. Motivated by the fact that a graph convolutional network (GCN) has been effective for graph-based SSL, we propose HyperGCN, a novel GCN for SSL on attributed hypergraphs. Additionally, we show how HyperGCN can be used as a learning-based approach for combinatorial optimisation on NP-hard hypergraph problems. We demonstrate HyperGCN's effectiveness through detailed experimentation on real-world hypergraphs.

[1]  Olgica Milenkovic,et al.  Quadratic Decomposable Submodular Function Minimization , 2018, NeurIPS.

[2]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[3]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[4]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[5]  Pan Li,et al.  HS2: Active learning over hypergraphs with pointwise and pairwise queries , 2019, AISTATS.

[6]  Olgica Milenkovic,et al.  Revisiting Decomposable Submodular Function Minimization with Incidence Relations , 2018, NeurIPS.

[7]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[8]  Yixin Chen,et al.  Beyond Link Prediction: Predicting Hyperlinks in Adjacency Space , 2018, AAAI.

[9]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[10]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[11]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[12]  Chenzi Zhang,et al.  Spectral Properties of Hypergraph Laplacian and Approximation Algorithms , 2016, J. ACM.

[13]  T.-H. Hubert Chan,et al.  Generalizing the Hypergraph Laplacian via a Diffusion Process with Mediators , 2018, COCOON.

[14]  Yue Gao,et al.  Hypergraph Neural Networks , 2018, AAAI.

[15]  Anand Louis,et al.  Hypergraph Markov Operators, Eigenvalues and Approximation Algorithms , 2014, STOC.

[16]  Olgica Milenkovic,et al.  Submodular Hypergraphs: p-Laplacians, Cheeger Inequalities and Spectral Clustering , 2018, ICML.

[17]  Michael Dinitz,et al.  The Densest k-Subhypergraph Problem , 2016, APPROX-RANDOM.

[18]  Marcello Pelillo,et al.  A Game-Theoretic Approach to Hypergraph Clustering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Yu Gong,et al.  Exact-K Recommendation via Maximal Clique Optimization , 2019, KDD.

[20]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[21]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[22]  My T. Thai,et al.  Approximate k-Cover in Hypergraphs: Efficient Algorithms, and Applications , 2019, ArXiv.

[23]  Balaraman Ravindran,et al.  Extended Discriminative Random Walk: A Hypergraph Approach to Multi-View Multi-Relational Transductive Learning , 2015, IJCAI.

[24]  Matthias Hein,et al.  The Total Variation on Hypergraphs - Learning on Hypergraphs Revisited , 2013, NIPS.

[25]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[26]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[27]  Bernhard Schölkopf,et al.  Cluster Kernels for Semi-Supervised Learning , 2002, NIPS.

[28]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[29]  Sarah Parisot,et al.  Learning Conditioned Graph Structures for Interpretable Visual Question Answering , 2018, NeurIPS.

[30]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[31]  Abhinav Gupta,et al.  Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Serge J. Belongie,et al.  Higher order learning with graphs , 2006, ICML.

[33]  Yiqun Liu,et al.  Learning on Partial-Order Hypergraphs , 2018, WWW.

[34]  Chenzi Zhang,et al.  Re-revisiting Learning on Hypergraphs: Confidence Interval and Subgradient Method , 2017, ICML.

[35]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[36]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[37]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Chenzi Zhang,et al.  Diffusion Operator and Spectral Analysis for Directed Hypergraph Laplacian , 2017, Theor. Comput. Sci..

[39]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[40]  Luís C. Lamb,et al.  Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems , 2019, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI).

[41]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[42]  Benjamin J Raphael,et al.  Random Walks on Hypergraphs with Edge-Dependent Vertex Weights , 2019, ICML.

[43]  Austin R. Benson,et al.  Planted Hitting Set Recovery in Hypergraphs , 2019, ArXiv.

[44]  Partha Pratim Talukdar,et al.  Graph-Based Semi-Supervised Learning , 2014, Graph-Based Semi-Supervised Learning.

[45]  Dan Alistarh,et al.  Powerset Convolutional Neural Networks , 2019, NeurIPS.

[46]  Olgica Milenkovic,et al.  Inhomogeneous Hypergraph Clustering with Applications , 2017, NIPS.

[47]  Chiranjib Bhattacharyya,et al.  Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks , 2018, ACL.

[48]  Prateek Yadav,et al.  Confidence-based Graph Convolutional Networks for Semi-Supervised Learning , 2019, AISTATS.

[49]  Fei Wang,et al.  Structural Deep Embedding for Hyper-Networks , 2017, AAAI.

[50]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[51]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[52]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[53]  Yue Gao,et al.  Dynamic Hypergraph Neural Networks , 2019, IJCAI.

[54]  Stephan Günnemann,et al.  Dual-Primal Graph Convolutional Networks , 2018, ArXiv.

[55]  Liujuan Cao,et al.  Hypergraph Induced Convolutional Manifold Networks , 2019, IJCAI.

[56]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[57]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[58]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[59]  Philip S. Yu,et al.  Adversarial Attack and Defense on Graph Data: A Survey , 2018 .

[60]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[61]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[62]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.