Fast, Warped Graph Embedding: Unifying Framework and One-Click Algorithm

What is the best way to describe a user in a social network with just a few numbers? Mathematically, this is equivalent to assigning a vector representation to each node in a graph, a process called graph embedding. We propose a novel framework, GEM-D that unifies most of the past algorithms such as LapEigs, DeepWalk and node2vec. GEM-D achieves its goal by decomposing any graph embedding algorithm into three building blocks: node proximity function, warping function and loss function. Based on thorough analysis of GEM-D, we propose a novel algorithm, called UltimateWalk, which outperforms the most-recently proposed state-of-the-art DeepWalk and node2vec. The contributions of this work are: (1) The proposed framework, GEM-D unifies the past graph embedding algorithms and provides a general recipe of how to design a graph embedding; (2) the nonlinearlity in the warping function contributes significantly to the quality of embedding and the exponential function is empirically optimal; (3) the proposed algorithm, UltimateWalk is one-click (no user-defined parameters), scalable and has a closed-form solution.

[1]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[2]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[3]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[4]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[5]  Ivan Herman,et al.  Graph Visualization and Navigation in Information Visualization: A Survey , 2000, IEEE Trans. Vis. Comput. Graph..

[6]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[7]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[8]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[9]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[10]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[11]  Jingzhou Liu,et al.  Visualizing Large-scale and High-dimensional Data , 2016, WWW.

[12]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[13]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[14]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[15]  Kevin Chen-Chuan Chang,et al.  From Node Embedding To Community Embedding , 2016, ArXiv.

[16]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[17]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[18]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[19]  Huan Liu,et al.  Scalable learning of collective behavior based on sparse social dimensions , 2009, CIKM.

[20]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[21]  Jure Leskovec,et al.  Supervised random walks: predicting and recommending links in social networks , 2010, WSDM '11.

[22]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[23]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[24]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[25]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[26]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[27]  Taher H. Haveliwala.  Topic-Sensitive PageRank: A Context-Sensitive Ranking Algorithm for Web Search , 2003, IEEE Trans. Knowl. Data Eng..

[28]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[29]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[30]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[31]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[32]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .