Graph Embedding for Combinatorial Optimization: A Survey

Graphs have been widely used to represent complex data in many applications, such as e-commerce, social networks, and bioinformatics. Efficient and effective analysis of graph data is important for graph-based applications. However, most graph analysis tasks are combinatorial optimization (CO) problems, which are NP-hard. Recent studies have focused a lot on the potential of using machine learning (ML) to solve graph-based CO problems. Using ML- based CO methods, a graph has to be represented in numerical vectors, which is known as graph embedding. In this survey, we provide a thorough overview of recent graph embedding methods that have been used to solve CO problems. Most graph embedding methods have two stages: graph preprocessing and ML model learning. This survey classifies graph embedding works from the perspective of graph preprocessing tasks and ML models. Furthermore, this survey summarizes recent graph-based CO methods that exploit graph embedding. In particular, graph embedding can be employed as part of classification techniques or can be combined with search methods to find solutions to CO problems. The survey ends with several remarks on future research directions.

[1]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[2]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[3]  Pushmeet Kohli,et al.  Graph Matching Networks for Learning the Similarity of Graph Structured Objects , 2019, ICML.

[4]  Subhash Khot,et al.  Improved inapproximability results for MaxClique, chromatic number and approximate graph coloring , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[5]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[6]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[7]  Tsuyoshi Murata,et al.  Motif-Aware Graph Embeddings , 2017 .

[8]  Jiawei Han,et al.  Node, Motif and Subgraph: Leveraging Network Functional Blocks Through Structural Convolution , 2018, 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[9]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[10]  Vachik S. Dave,et al.  Neural-Brane: Neural Bayesian Personalized Ranking for Attributed Network Embedding , 2018, Data Science and Engineering.

[11]  Yun Peng,et al.  VColor: A practical vertex-cut based approach for coloring large graphs , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[12]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[13]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[14]  Xiaokui Xiao,et al.  Homogeneous network embedding for massive graphs via reweighted personalized PageRank , 2019, Proc. VLDB Endow..

[15]  Azalia Mirhoseini,et al.  GAP: Generalizable Approximate Graph Partitioning Framework , 2019, ArXiv.

[16]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[17]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[18]  Manoj Reddy Dareddy,et al.  motif2vec: Motif Aware Node Representation Learning for Heterogeneous Networks , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[19]  Yizhou Sun,et al.  Fast Detection of Maximum Common Subgraph via Deep Q-Learning , 2020, ArXiv.

[20]  Jiawei Zhang,et al.  IsoNN: Isomorphic Neural Network for Graph Representation Learning and Classification , 2019, ArXiv.

[21]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[22]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[23]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[24]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[25]  Chang Zhou,et al.  Scalable Graph Embedding for Asymmetric Proximity , 2017, AAAI.

[26]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[27]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[28]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[29]  A. Stephen McGough,et al.  Exploring the Semantic Content of Unsupervised Graph Embeddings: An Empirical Study , 2018, Data Science and Engineering.

[30]  Santosh S. Vempala,et al.  On The Approximability Of The Traveling Salesman Problem , 2006, Comb..

[31]  Steven Skiena,et al.  HARP: Hierarchical Representation Learning for Networks , 2017, AAAI.

[32]  C.-C. Jay Kuo,et al.  Graph representation learning: a survey , 2019, APSIPA Transactions on Signal and Information Processing.

[33]  Evgeny Burnaev,et al.  Reinforcement Learning for Combinatorial Optimization: A Survey , 2020, ArXiv.

[34]  Yun Peng,et al.  Authenticated Subgraph Similarity Searchin Outsourced Graph Databases , 2015, IEEE Transactions on Knowledge and Data Engineering.

[35]  Emmanuel Müller,et al.  VERSE: Versatile Graph Embeddings from Similarity Measures , 2018, WWW.

[36]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[37]  Christopher R'e,et al.  Machine Learning on Graphs: A Model and Comprehensive Taxonomy , 2020, ArXiv.

[38]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[39]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[40]  Stanley Wasserman,et al.  Social Network Analysis: Methods and Applications , 1994 .

[41]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[42]  Esra Akbas,et al.  Network Embedding: on Compression and Learning , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[43]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[44]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[45]  Yizhou Sun,et al.  SimGNN: A Neural Network Approach to Fast Graph Similarity Computation , 2018, WSDM.

[46]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[47]  Wen-Chih Peng,et al.  Exploiting Centrality Information with Graph Convolutions for Network Representation Learning , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[48]  Uzi Vishkin,et al.  Solving NP-Hard Problems on Graphs That Are Almost Trees and an Application to Facility Location Problems , 1984, JACM.

[49]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[50]  Zhiwu Lu,et al.  RUM: Network Representation Learning Using Motifs , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[51]  Luís C. Lamb,et al.  Learning to Solve NP-Complete Problems - A Graph Neural Network for the Decision TSP , 2018, AAAI.

[52]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[53]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[54]  Yang Liu,et al.  subgraph2vec: Learning Distributed Representations of Rooted Sub-graphs from Large Graphs , 2016, ArXiv.

[55]  Richard Peng,et al.  Faster Graph Embeddings via Coarsening , 2020, ICML.

[56]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[57]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[58]  Lijun Chang,et al.  Efficient Maximum Clique Computation over Large Sparse Graphs , 2019, KDD.

[59]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[60]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[61]  Md. Mostofa Ali Patwary,et al.  Coloring Big Graphs with AlphaGoZero , 2019, ArXiv.

[62]  J. Håstad Clique is hard to approximate withinn1−ε , 1999 .

[63]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[64]  Srinivasan Parthasarathy,et al.  Graph embedding on biomedical networks: methods, applications and evaluations , 2019, Bioinform..

[65]  Hisashi Kashima,et al.  Approximation Ratios of Graph Neural Networks for Combinatorial Problems , 2019, NeurIPS.

[66]  Kate Smith-Miles,et al.  Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research , 1999, INFORMS J. Comput..

[67]  Kate A. Smith,et al.  Neural Networks for Combinatorial Optimization: a Review of More Than a Decade of Research , 1999 .

[68]  Senzhang Wang,et al.  Motif-Matching Based Subgraph-Level Attentional Convolutional Network for Graph Classification , 2020, AAAI.

[69]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Dacheng Tao,et al.  SPAGAN: Shortest Path Graph Attention Network , 2019, IJCAI.

[71]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[72]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[73]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[74]  Laks V. S. Lakshmanan,et al.  Community Search over Big Graphs , 2019, Synthesis Lectures on Data Management.

[75]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[76]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[77]  Yun Peng,et al.  Towards Efficient Authenticated Subgraph Query Service in Outsourced Graph Databases , 2014, IEEE Transactions on Services Computing.

[78]  Yizhou Sun,et al.  Learning-Based Efficient Graph Similarity Computation via Multi-Scale Convolutional Set Matching , 2020, AAAI.

[79]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[80]  Junzhou Huang,et al.  Adaptive Sampling Towards Fast Graph Representation Learning , 2018, NeurIPS.

[81]  Irwin King,et al.  MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding , 2020, WWW.