Geometric Laplacian Eigenmap Embedding

Graph embedding seeks to build a low-dimensional representation of a graph G. This low-dimensional representation is then used for various downstream tasks. One popular approach is Laplacian Eigenmaps, which constructs a graph embedding based on the spectral properties of the Laplacian matrix of G. The intuition behind it, and many other embedding techniques, is that the embedding of a graph must respect node similarity: similar nodes must have embeddings that are close to one another. Here, we dispose of this distance-minimization assumption. Instead, we use the Laplacian matrix to find an embedding with geometric properties instead of spectral ones, by leveraging the so-called simplex geometry of G. We introduce a new approach, Geometric Laplacian Eigenmap Embedding (or GLEE for short), and demonstrate that it outperforms various other techniques (including Laplacian Eigenmaps) in the tasks of graph reconstruction and link prediction.

[1]  Christos Faloutsos,et al.  Fast, Warped Graph Embedding: Unifying Framework and One-Click Algorithm , 2017, ArXiv.

[2]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[3]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[4]  Balasubramaniam Srinivasan,et al.  On the Equivalence between Node Embeddings and Structural Graph Representations , 2019, ICLR 2020.

[5]  Albert-László Barabási,et al.  Network-based prediction of protein interactions , 2018, Nature Communications.

[6]  Jure Leskovec,et al.  Predicting positive and negative links in online social networks , 2010, WWW '10.

[7]  Christos Faloutsos,et al.  Efficiently spotting the starting points of an epidemic in a large graph , 2013, Knowledge and Information Systems.

[8]  Hanghang Tong,et al.  Fast Eigen-Functions Tracking on Dynamic Graphs , 2015, SDM.

[9]  B. Bollobás.  The evolution of random graphs , 1984 .

[10]  Danai Koutra,et al.  Latent Network Summarization: Bridging Network Embedding and Summarization , 2018, KDD.

[11]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[12]  Piet Van Mieghem,et al.  Graph Spectra for Complex Networks , 2010 .

[13]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[14]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[15]  A. Rbnyi.  ON THE EVOLUTION OF RANDOM GRAPHS , 2001 .

[16]  Daniel A. Spielman,et al.  Graphs, Vectors, and Matrices , 2016 .

[17]  Hanghang Tong,et al.  On the eigen‐functions of dynamic graphs: Fast tracking and attribution algorithms , 2017, Stat. Anal. Data Min..

[18]  Carey E. Priebe,et al.  Out-of-sample extension of graph adjacency spectral embedding , 2018, ICML.

[19]  Piet Van Mieghem,et al.  The Simplex Geometry of Graphs , 2018, J. Complex Networks.

[20]  Chris H. Q. Ding,et al.  Symmetric Nonnegative Matrix Factorization for Graph Clustering , 2012, SDM.

[21]  Piet Van Mieghem,et al.  An upper bound for the epidemic threshold in exact Markovian SIR and SIS epidemics on networks , 2014, 53rd IEEE Conference on Decision and Control.

[22]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[23]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[24]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[25]  Miroslav Fiedler,et al.  Matrices and Graphs in Geometry , 2011 .

[26]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[27]  Y. Koren,et al.  Drawing graphs by eigenvectors: theory and practice , 2005 .

[28]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[29]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[30]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[31]  Marián Boguñá,et al.  Popularity versus similarity in growing networks , 2011, Nature.

[32]  Ernesto Estrada,et al.  Hyperspherical embedding of graphs and networks in communicability spaces , 2014, Discret. Appl. Math..

[33]  W. Zachary,et al.  An Information Flow Model for Conflict and Fission in Small Groups , 1977, Journal of Anthropological Research.

[34]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[35]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[36]  Piet Van Mieghem,et al.  On the Robustness of Complex Networks by Using the Algebraic Connectivity , 2008, Networking.

[37]  Austin R. Benson,et al.  Incrementally Updated Spectral Embeddings , 2019, ArXiv.

[38]  John Shawe-Taylor,et al.  Characterizing Graph Drawing with Eigenvectors , 2000, J. Chem. Inf. Comput. Sci..

[39]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[41]  Ernesto Estrada,et al.  Machine Learning Analysis of Complex Networks in Hyperspherical Space , 2018, 1804.05960.

[42]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[43]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[44]  Shreyas Sundaram,et al.  Robustness and Algebraic Connectivity of Random Interdependent Networks , 2015, ArXiv.

[45]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[46]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[47]  Purnamrita Sarkar,et al.  Theoretical Justification of Popular Link Prediction Heuristics , 2011, IJCAI.

[48]  Bridget E. Begg,et al.  A Proteome-Scale Map of the Human Interactome Network , 2014, Cell.

[49]  Steven Skiena,et al.  A Tutorial on Network Embeddings , 2018, ArXiv.

[50]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[51]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.