Hashing-Accelerated Graph Neural Networks for Link Prediction

Networks are ubiquitous in the real world. Link prediction, as one of the key problems for network-structured data, aims to predict whether there exists a link between two nodes. The traditional approaches are based on the explicit similarity computation between the compact node representation by embedding each node into a low-dimensional space. In order to efficiently handle the intensive similarity computation in link prediction, the hashing technique has been successfully used to produce the node representation in the Hamming space. However, the hashing-based link prediction algorithms face accuracy loss from the randomized hashing techniques or inefficiency from the learning to hash techniques in the embedding process. Currently, the Graph Neural Network (GNN) framework has been widely applied to the graph-related tasks in an end-to-end manner, but it commonly requires substantial computational resources and memory costs due to massive parameter learning, which makes the GNN-based algorithms impractical without the help of a powerful workhorse. In this paper, we propose a simple and effective model called #GNN, which balances the trade-off between accuracy and efficiency. #GNN is able to efficiently acquire node representation in the Hamming space for link prediction by exploiting the randomized hashing technique to implement message passing and capture high-order proximity in the GNN framework. Furthermore, we characterize the discriminative power of #GNN in probability. The extensive experimental results demonstrate that the proposed #GNN algorithm achieves accuracy comparable to the learning-based algorithms and outperforms the randomized algorithm, while running significantly faster than the learning-based algorithms. Also, the proposed algorithm shows excellent scalability on a large-scale network with the limited resources.

[1]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[2]  Chengqi Zhang,et al.  Consistent Weighted Sampling Made More Practical , 2017, WWW.

[3]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[4]  Chengqi Zhang,et al.  Efficient Attributed Network Embedding via Recursive Randomized Hashing , 2018, IJCAI.

[5]  Wu-Jun Li,et al.  Scalable Graph Hashing with Feature Transformation , 2015, IJCAI.

[6]  Chuan Zhou,et al.  Low-Bit Quantization for Attributed Network Representation Learning , 2019, IJCAI.

[7]  Jure Leskovec,et al.  Position-aware Graph Neural Networks , 2019, ICML.

[8]  Alan M. Frieze,et al.  Min-Wise Independent Permutations , 2000, J. Comput. Syst. Sci..

[9]  Lada A. Adamic,et al.  Friends and neighbors on the Web , 2003, Soc. Networks.

[10]  Gurmeet Singh Manku,et al.  Detecting near-duplicates for web crawling , 2007, WWW '07.

[11]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[12]  Antonio Torralba,et al.  Spectral Hashing , 2008, NIPS.

[13]  Chengqi Zhang,et al.  Homophily, Structure, and Content Augmented Network Representation Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[14]  Paolo Rosso,et al.  NodeSketch: Highly-Efficient Graph Embeddings via Recursive Sketching , 2019, KDD.

[15]  Jure Leskovec,et al.  Learning to Discover Social Circles in Ego Networks , 2012, NIPS.

[16]  Jianmin Wang,et al.  Flexible Attributed Network Embedding , 2018, ArXiv.

[17]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[18]  Chengqi Zhang,et al.  $K$ -Ary Tree Hashing for Fast Graph Classification , 2018, IEEE Transactions on Knowledge and Data Engineering.

[19]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[20]  Regina Barzilay,et al.  Deriving Neural Architectures from Sequence and Graph Kernels , 2017, ICML.

[21]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[22]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[23]  Philip S. Yu,et al.  Improved Consistent Weighted Sampling Revisited , 2017, IEEE Transactions on Knowledge and Data Engineering.

[24]  Yixin Chen,et al.  Weisfeiler-Lehman Neural Machine for Link Prediction , 2017, KDD.

[25]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[26]  Alan M. Frieze,et al.  Min-wise independent permutations (extended abstract) , 1998, STOC '98.

[27]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[28]  Xing Zhao,et al.  Learning to Hash with Graph Neural Networks for Recommender Systems , 2020, WWW.

[29]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[30]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[31]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[32]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[33]  Kilian Q. Weinberger,et al.  Feature hashing for large scale multitask learning , 2009, ICML '09.

[34]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[35]  Evangelos Kalogerakis,et al.  SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Chengqi Zhang,et al.  Nested Subtree Hash Kernels for Large-Scale Graph Classification over Streams , 2012, 2012 IEEE 12th International Conference on Data Mining.

[37]  Lin Yang,et al.  Asymmetric Discrete Graph Hashing , 2017, AAAI.

[38]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[39]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[40]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[41]  Bin Li,et al.  A Review for Weighted MinHash Algorithms , 2018, IEEE Transactions on Knowledge and Data Engineering.

[42]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[43]  Vijay S. Pande,et al.  Molecular graph convolutions: moving beyond fingerprints , 2016, Journal of Computer-Aided Molecular Design.

[44]  Geoffrey E. Hinton,et al.  Semantic hashing , 2009, Int. J. Approx. Reason..

[45]  Charu C. Aggarwal,et al.  On Classification of Graph Streams , 2011, SDM.

[46]  Yixin Chen,et al.  BoostGAPFILL: improving the fidelity of metabolic network reconstructions through integrated constraint and pattern‐based methods , 2016, Bioinform..

[47]  Moses Charikar,et al.  Similarity estimation techniques from rounding algorithms , 2002, STOC '02.

[48]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[49]  Chengqi Zhang,et al.  Canonical Consistent Weighted Sampling for Real-Value Weighted Min-Hash , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[50]  Dan Xu,et al.  Dynamic Graph Message Passing Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[52]  K. Ramanan,et al.  Concentration Inequalities for Dependent Random Variables via the Martingale Method , 2006, math/0609835.

[53]  Wei Liu,et al.  Discrete Graph Hashing , 2014, NIPS.

[54]  Chengqi Zhang,et al.  Binarized attributed network embedding , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[55]  Huan Liu,et al.  Unsupervised Streaming Feature Selection in Social Media , 2015, CIKM.

[56]  Zhiyuan Liu,et al.  CANE: Context-Aware Network Embedding for Relation Modeling , 2017, ACL.

[57]  KleinbergJon,et al.  The link-prediction problem for social networks , 2007 .

[58]  Xing Xie,et al.  High-order Proximity Preserving Information Network Hashing , 2018, KDD.