Semi-supervised Network Embedding with Differentiable Deep Quantisation

Learning accurate low-dimensional embeddings for a network is a crucial task as it facilitates many downstream network analytics tasks. For large networks, the trained embeddings often require a significant amount of space to store, making storage and processing a challenge. Building on our previous work on semi-supervised network embedding, we develop dSNEQ, a differentiable DNN-based quantisation method for network embedding. d-SNEQ incorporates a rank loss to equip the learned quantisation codes with rich high-order information, and is able to substantially compress the size of trained embeddings, thus reducing storage footprint and accelerating retrieval speed. We also propose a new evaluation metric, path prediction, to fairly and more directly evaluate model performance on the preservation of high-order information. Our evaluation on four real-world networks of diverse characteristics shows that d-SNEQ outperforms a number of state-of-the-art embedding methods in link prediction, path prediction, node classification and node recommendation, while being far more spaceand time-efficient.

[1]  Ludovic Dos Santos,et al.  Multilabel Classification on Heterogeneous Graphs with Gaussian Embeddings , 2016, ECML/PKDD.

[2]  Jian Sun,et al.  Optimized Product Quantization for Approximate Nearest Neighbor Search , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jianmin Wang,et al.  Deep Visual-Semantic Quantization for Efficient Image Retrieval , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Hideki Nakayama,et al.  Compressing Word Embeddings via Deep Compositional Code Learning , 2017, ICLR.

[5]  Xiao Wang,et al.  Hierarchical Taxonomy Aware Network Embedding , 2018, KDD.

[6]  Jian Pei,et al.  Arbitrary-Order Proximity Preserved Network Embedding , 2018, KDD.

[7]  Bin Liu,et al.  Deep Triplet Quantization , 2018, ACM Multimedia.

[8]  Jian Pei,et al.  High-Order Proximity Preserved Embedding for Dynamic Networks , 2018, IEEE Transactions on Knowledge and Data Engineering.

[9]  M. Narasimha Murty,et al.  Outlier Aware Network Embedding for Attributed Networks , 2018, AAAI.

[10]  Jingkuan Song,et al.  Binary Generative Adversarial Networks for Image Retrieval , 2017, AAAI.

[11]  Artem Babenko,et al.  Unsupervised Neural Quantization for Compressed-Domain Similarity Search , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Nicu Sebe,et al.  A Survey on Learning to Hash , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Lina Yao,et al.  Adversarially Regularized Graph Autoencoder , 2018, IJCAI.

[14]  Chengqi Zhang,et al.  Binarized attributed network embedding , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[15]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[16]  Victor Lempitsky,et al.  Additive Quantization for Extreme Vector Compression , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Minyi Guo,et al.  Supervised hashing with latent factor models , 2014, SIGIR.

[18]  Jianfei Cai,et al.  Scene Graph Generation With External Knowledge and Image Reconstruction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[20]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[21]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[22]  Wu-Jun Li,et al.  Feature Learning Based Deep Supervised Hashing with Pairwise Labels , 2015, IJCAI.

[23]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, Knowledge and Information Systems.

[24]  Srinivasan Parthasarathy,et al.  SEANO: Semi-supervised Embedding in Attributed Networks with Outliers , 2017, SDM.

[25]  Zhiyuan Liu,et al.  Fast Network Embedding Enhancement via High Order Proximity Approximation , 2017, IJCAI.

[26]  Ping Lu,et al.  Person re-identification by enhanced local maximal occurrence representation and generalized similarity metric learning , 2018, Neurocomputing.

[27]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[29]  Chengqi Zhang,et al.  Efficient Attributed Network Embedding via Recursive Randomized Hashing , 2018, IJCAI.

[30]  Jianmin Wang,et al.  Deep Quantization Network for Efficient Image Retrieval , 2016, AAAI.

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Cheng Wang,et al.  Approximate Nearest Neighbor Search by Residual Vector Quantization , 2010, Sensors.

[33]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[34]  Yejin Choi,et al.  Neural Motifs: Scene Graph Parsing with Global Context , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[36]  Jingdong Wang,et al.  Inner Product Similarity Search using Compositional Codes , 2014, ArXiv.

[37]  Wei Chen,et al.  Galaxy Network Embedding: A Hierarchical Community Structure Preserving Approach , 2018, IJCAI.

[38]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[39]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Xing Xie,et al.  High-order Proximity Preserving Information Network Hashing , 2018, KDD.

[41]  Cordelia Schmid,et al.  Product Quantization for Nearest Neighbor Search , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[43]  Nicholay Topin,et al.  Super-convergence: very fast training of neural networks using large learning rates , 2018, Defense + Commercial Sensing.

[44]  Piotr Sankowski,et al.  Shortest Paths in Matrix Multiplication Time , 2005, ESA.

[45]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[46]  Philip S. Yu,et al.  Deep Visual-Semantic Hashing for Cross-Modal Retrieval , 2016, KDD.

[47]  Xin Wang,et al.  SNEQ: Semi-Supervised Attributed Network Embedding with Attention-Based Quantisation , 2020, AAAI.

[48]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[49]  Chuan Chen,et al.  Learning Semantic Representations for Unsupervised Domain Adaptation , 2018, ICML.

[50]  Jiwen Lu,et al.  Deep hashing for compact binary codes learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Junsong Yuan,et al.  Product Quantization Network for Fast Image Retrieval , 2018, ECCV.

[52]  Yifei Zhang,et al.  Doc2hash: Learning Discrete Latent variables for Documents Retrieval , 2019, NAACL.

[53]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[54]  Tieniu Tan,et al.  Hierarchical Graph Convolutional Networks for Semi-supervised Node Classification , 2019, IJCAI.

[55]  Yao Yang,et al.  Triplet Enhanced AutoEncoder: Model-free Discriminative Network Embedding , 2019, IJCAI.

[56]  Ling Shao,et al.  Auto-Encoding Twin-Bottleneck Hashing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Steven Skiena,et al.  HARP: Hierarchical Representation Learning for Networks , 2017, AAAI.

[58]  David J. Fleet,et al.  Cartesian K-Means , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Huanbo Luan,et al.  Discrete Collaborative Filtering , 2016, SIGIR.

[60]  Stephan Günnemann,et al.  Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking , 2017, ICLR.