Graph Convolutional Networks for Graphs Containing Missing Features

Graph Convolutional Network (GCN) has experienced great success in graph analysis tasks. It works by smoothing the node features across the graph. The current GCN models overwhelmingly assume that node feature information is complete. However, real-world graph data are often incomplete and containing missing features. Traditionally, people have to estimate and fill in the unknown features based on imputation techniques and then apply GCN. However, the process of feature filling and graph learning are separated, resulting in degraded and unstable performance. This problem becomes more serious when a large number of features are missing. We propose an approach that adapts GCN to graphs containing missing features. In contrast to traditional strategy, our approach integrates the processing of missing features and graph learning within the same neural network architecture. Our idea is to represent the missing data by Gaussian Mixture Model (GMM) and calculate the expected activation of neurons in the first hidden layer of GCN, while keeping the other layers of the network unchanged. This enables us to learn the GMM parameters and network weight parameters in an end-to-end manner. Notably, our approach does not increase the computational complexity of GCN and it is consistent with GCN when the features are complete. We conduct experiments on the node label classification task and demonstrate that our approach significantly outperforms the best imputation based methods by up to 99.43%, 102.96%, 6.97%, 35.36% in four benchmark graphs when a large portion of features are missing. The performance of our approach for the case with a low level of missing features is even superior to GCN for the case with complete features.

[1]  Si Zhang,et al.  Graph convolutional networks: a comprehensive review , 2019, Computational Social Networks.

[2]  Tsuyoshi Murata,et al.  Fast Approximations of Betweenness Centrality with Graph Neural Networks , 2019, CIKM.

[3]  Xiaochun Cao,et al.  Topology Optimization based Graph Convolutional Network , 2019, IJCAI.

[4]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[5]  Nikos Komodakis,et al.  Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Philip S. Yu,et al.  Graph Learning Approaches to Recommender Systems: A Review , 2020, ArXiv.

[7]  Jacek Tabor,et al.  Generalized RBF kernel for incomplete data , 2016, Knowl. Based Syst..

[8]  S. Hewitt,et al.  1977 , 1977, Kuwait 1975/76 - 2019.

[9]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[10]  Ying Zhang,et al.  Multivariate Time Series Imputation with Generative Adversarial Networks , 2018, NeurIPS.

[11]  蕭瓊瑞撰述,et al.  2009 , 2019, The Winning Cars of the Indianapolis 500.

[12]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[13]  Jure Leskovec,et al.  Modeling polypharmacy side effects with graph convolutional networks , 2018, bioRxiv.

[14]  S. Hewitt,et al.  2007 , 2018, Los 25 años de la OMC: Una retrospectiva fotográfica.

[15]  Wen-Chih Peng,et al.  Exploiting Centrality Information with Graph Convolutions for Network Representation Learning , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[16]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[17]  Alex Fout,et al.  Protein Interface Prediction using Graph Convolutional Networks , 2017, NIPS.

[18]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[19]  Khalil Sima'an,et al.  Graph Convolutional Encoders for Syntax-aware Neural Machine Translation , 2017, EMNLP.

[20]  Philip S. Yu,et al.  Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting , 2020, Inf. Sci..

[21]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[22]  Aníbal R. Figueiras-Vidal,et al.  Pattern classification with missing data: a review , 2010, Neural Computing and Applications.

[23]  Jianxun Liu,et al.  Feature-Attention Graph Convolutional Networks for Noise Resilient Learning , 2019, ArXiv.

[24]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[25]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[26]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[27]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[28]  Takuya Akiba,et al.  Optuna: A Next-generation Hyperparameter Optimization Framework , 2019, KDD.

[29]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[30]  Philip S. Yu,et al.  A Survey on Knowledge Graphs: Representation, Acquisition, and Applications , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Simone Scardapane,et al.  Missing Data Imputation with Adversarially-trained Graph Convolutional Networks , 2019, Neural Networks.

[32]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[33]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[34]  Peter Bühlmann,et al.  MissForest - non-parametric missing value imputation for mixed-type data , 2011, Bioinform..

[35]  Tao Mei,et al.  Exploring Visual Relationship for Image Captioning , 2018, ECCV.

[36]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[37]  Thomas Hofmann,et al.  Kernel Methods for Missing Variables , 2005, AISTATS.

[38]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[39]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .

[40]  Yizhou Sun,et al.  SimGNN: A Neural Network Approach to Fast Graph Similarity Computation , 2018, WSDM.

[41]  Regina Barzilay,et al.  GraphIE: A Graph-Based Framework for Information Extraction , 2018, NAACL.

[42]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[43]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[44]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[45]  Ke Wang,et al.  MIDA: Multiple Imputation Using Denoising Autoencoders , 2017, PAKDD.

[46]  L. Freeman,et al.  The Development of Social Network Analysis: A Study in the Sociology of Science , 2005 .

[47]  Tsuyoshi Murata,et al.  A General View for Network Embedding as Matrix Factorization , 2019, WSDM.

[48]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[49]  Amin Zheng,et al.  RGCNN: Regularized Graph CNN for Point Cloud Segmentation , 2018, ACM Multimedia.

[50]  Wenwu Zhu,et al.  Robust Graph Convolutional Networks Against Adversarial Attacks , 2019, KDD.

[51]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[52]  Xiaojun Chang,et al.  Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks , 2020, KDD.

[53]  Xiaochun Cao,et al.  Graph Attention Topic Modeling Network , 2020, WWW.

[54]  Tsuyoshi Murata,et al.  Learning Community Structure with Variational Autoencoder , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[55]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[56]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[57]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[58]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[59]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[60]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[61]  양희영.  2005 , 2005, Los 25 años de la OMC: Una retrospectiva fotográfica.

[62]  Dongxiao He,et al.  Community Detection via Joint Graph Convolutional Network Embedding in Attribute Network , 2019, ICANN.

[63]  Philip S. Yu,et al.  Alleviating the Inconsistency Problem of Applying Graph Neural Network to Fraud Detection , 2020, SIGIR.

[64]  Weixiong Zhang,et al.  Graph Convolutional Networks Meet Markov Random Fields: Semi-Supervised Community Detection in Attribute Networks , 2019, AAAI.

[65]  Shirui Pan,et al.  Unsupervised Domain Adaptive Graph Convolutional Networks , 2020, WWW.

[66]  Yi-Cheng Zhang,et al.  Bipartite network projection and personal recommendation. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[67]  Kai Jiang,et al.  Classification for Incomplete Data Using Classifier Ensembles , 2005, 2005 International Conference on Neural Networks and Brain.

[68]  Philip S. Yu,et al.  Edge2vec: Edge-based Social Network Embedding , 2020, ACM Trans. Knowl. Discov. Data.

[69]  Yizhou Sun,et al.  Learning to Identify High Betweenness Centrality Nodes from Scratch: A Novel Graph Neural Network Approach , 2019, CIKM.

[70]  Liming Zhu,et al.  Going Deep: Graph Convolutional Ladder-Shape Networks , 2020, AAAI.

[71]  Stephan Günnemann,et al.  Certifiable Robustness and Robust Training for Graph Convolutional Networks , 2019, KDD.

[72]  Jure Leskovec,et al.  Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems , 2019, KDD.

[73]  Eunho Yang,et al.  Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks , 2019, ICLR.

[74]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[75]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[76]  Kaleem Siddiqi,et al.  Local Spectral Graph Convolution for Point Set Feature Learning , 2018, ECCV.

[77]  Johan A. K. Suykens,et al.  Handling missing values in support vector machine classifiers , 2005, Neural Networks.

[78]  Svetlana Lazebnik,et al.  Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering , 2018, NeurIPS.

[79]  Jing Jiang,et al.  Attributed Graph Clustering: A Deep Attentional Embedding Approach , 2019, IJCAI.

[80]  Kristina Lerman,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[81]  Kristie B. Hadden,et al.  2020 , 2020, Journal of Surgical Orthopaedic Advances.

[82]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[83]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[84]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[85]  A. Barabasi,et al.  Network medicine : a network-based approach to human disease , 2010 .

[86]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[87]  Takuya Akiba,et al.  Optuna , 2019, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

[88]  Gustavo E. A. P. A. Batista,et al.  A Study of K-Nearest Neighbour as an Imputation Method , 2002, HIS.

[89]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[90]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[91]  Chengqi Zhang,et al.  Learning Graph Embedding With Adversarial Training Methods , 2019, IEEE Transactions on Cybernetics.

[92]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[93]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[94]  Diego Marcheggiani,et al.  Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks , 2018, NAACL.

[95]  Le Wu,et al.  A Neural Influence Diffusion Model for Social Recommendation , 2019, SIGIR.

[96]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[97]  Weiwei Liu,et al.  Discrete Network Embedding , 2018, IJCAI.

[98]  Jacek Tabor,et al.  Processing of missing data by neural networks , 2018, NeurIPS.

[99]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[100]  Lawrence Carin,et al.  Incomplete-data classification using logistic regression , 2005, ICML.

[101]  Feng Xia,et al.  Big Networks: A Survey , 2020, Comput. Sci. Rev..

[102]  Stephan Günnemann,et al.  Adversarial Attacks on Neural Networks for Graph Data , 2018, KDD.

[103]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[104]  Olaf Sporns,et al.  The Human Connectome: A Structural Description of the Human Brain , 2005, PLoS Comput. Biol..

[105]  Lei Pan,et al.  Domain-Adversarial Graph Neural Networks for Text Classification , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[106]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[107]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[108]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[109]  D. Rubin.  Multiple imputation for nonresponse in surveys , 1989 .

[110]  Mihaela van der Schaar,et al.  GAIN: Missing Data Imputation using Generative Adversarial Nets , 2018, ICML.

[111]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[112]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[113]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..