Query-Specific Deep Embedding of Content-Rich Network

In this paper, we propose to embed a content-rich network for the purpose of similarity searching for a query node. In this network, besides the information of the nodes and edges, we also have the content of each node. We use the convolutional neural network (CNN) to represent the content of each node and then use the graph convolutional network (GCN) to further represent the node by merging the representations of its neighboring nodes. The GCN output is further fed to a deep encoder-decoder model to convert each node to a Gaussian distribution and then convert back to its node identity. The dissimilarity between the two nodes is measured by the Wasserstein distance between their Gaussian distributions. We define the nodes of the network to be positives if they are relevant to the query node and negative if they are irrelevant. The labeling of the positives/negatives is based on an upper bound and a lower bound of the Wasserstein distances between the candidate nodes and the query nodes. We learn the parameters of CNN, GCN, encoder-decoder model, Gaussian distributions, and the upper bound and lower bounds jointly. The learning problem is modeled as a minimization problem to minimize the losses of node identification, network structure preservation, positive/negative query-specific relevance-guild distance, and model complexity. An iterative algorithm is developed to solve the minimization problem. We conducted experiments over benchmark networks, especially innovation networks, to verify the effectiveness of the proposed method and showed its advantage over the state-of-the-art methods.

[1]  John Hughes,et al.  Generating Better Search Engine Text Advertisements with Deep Reinforcement Learning , 2019, KDD.

[2]  Stephan Gunnemann,et al.  Certifiable Robustness and Robust Training for Graph Convolutional Networks , 2019, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

[3]  Chin-Hui Lee,et al.  A maximal figure-of-merit learning approach to maximizing mean average precision with deep neural network based classifiers , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Wenwu Zhu,et al.  Deep Variational Network Embedding in Wasserstein Space , 2018, KDD.

[5]  Sándor Dominich,et al.  PageRank and Interaction Information Retrieval , 2005, J. Assoc. Inf. Sci. Technol..

[6]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Hayaru Shouno,et al.  Analysis of function of rectified linear unit used in deep learning , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[8]  Jun Zhao,et al.  Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks , 2015, ACL.

[9]  Qiang Li,et al.  Type Sequence Preserving Heterogeneous Information Network Embedding , 2019, AAAI.

[10]  Abien Fred Agarap Deep Learning using Rectified Linear Units (ReLU) , 2018, ArXiv.

[11]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[12]  Hanghang Tong,et al.  QUINT: On Query-Specific Optimal Networks , 2016, KDD.

[13]  Paolo Rosso,et al.  NodeSketch: Highly-Efficient Graph Embeddings via Recursive Sketching , 2019, KDD.

[14]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[15]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[16]  Philip S. Yu,et al.  Deep Recursive Network Embedding with Regular Equivalence , 2018, KDD.

[17]  Hong-Kun Xu,et al.  Strong convergence of the CQ method for fixed point iteration processes , 2006 .

[18]  Qiong Wu,et al.  Improving Deep Neural Network with Multiple Parametric Exponential Linear Units , 2016, Neurocomputing.

[19]  Hongxia Yang,et al.  Is a Single Vector Enough?: Exploring Node Polysemy for Network Embedding , 2019, KDD.

[20]  Ming Zhou,et al.  Question Answering over Freebase with Multi-Column Convolutional Neural Networks , 2015, ACL.

[21]  Hao Chen,et al.  Content-rich biological network constructed by mining PubMed abstracts , 2004, BMC Bioinformatics.

[22]  Brahim Chaib-draa,et al.  Parametric Exponential Linear Unit for Deep Convolutional Neural Networks , 2016, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[23]  Yue Cao,et al.  Transferable Representation Learning with Deep Adaptation Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Wray L. Buntine,et al.  Bibliographic Analysis with the Citation Network Topic Model , 2016, ACML.

[25]  Anish Shah,et al.  Deep Residual Networks with Exponential Linear Unit , 2016, ArXiv.

[26]  Evgenija D. Popova Generalization of a Parametric Fixed‐Point Iteration , 2004 .

[27]  Zheng Wang,et al.  Effective and Efficient Sports Play Retrieval with Deep Representation Learning , 2019, KDD.

[28]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[29]  Tie-Yan Liu Learning to Rank for Information Retrieval , 2009, Found. Trends Inf. Retr..

[30]  Vittorio Ferrari,et al.  End-to-End Training of Object Class Detectors for Mean Average Precision , 2016, ACCV.

[31]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[32]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[33]  Danai Koutra,et al.  Latent Network Summarization: Bridging Network Embedding and Summarization , 2018, KDD.

[34]  S. Ishikawa Fixed points by a new iteration method , 1974 .

[35]  Jian Pei,et al.  ProGAN: Network Embedding via Proximity Generative Adversarial Network , 2019, KDD.

[36]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[37]  Zhuowen Tu,et al.  Learning Context-Sensitive Shape Similarity by Graph Transduction , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[39]  Sahin Cem Geyik,et al.  Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search , 2019, KDD.

[40]  Charu C. Aggarwal,et al.  Graph Convolutional Networks with EigenPooling , 2019, KDD.

[41]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[42]  Carey E. Priebe,et al.  Joint Embedding of Graphs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Longin Jan Latecki,et al.  Affinity Learning with Diffusion on Tensor Product Graph , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Andrew McCallum,et al.  Automating the Construction of Internet Portals with Machine Learning , 2000, Information Retrieval.

[45]  B. Rhoades,et al.  Comments on two fixed point iteration methods , 1976 .

[46]  Jian Pei,et al.  Conditional Random Field Enhanced Graph Convolutional Neural Networks , 2019, KDD.

[47]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.