Graph Autoencoders with Deconvolutional Networks

Recent studies have indicated that Graph Convolutional Networks (GCNs) act as a low pass filter in spectral domain and encode smoothed node representations. In this paper, we consider their opposite, namely Graph Deconvolutional Networks (GDNs) that reconstruct graph signals from smoothed node representations. We motivate the design of Graph Deconvolutional Networks via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a high pass filter and may amplify the noise. Based on the proposed GDN, we further propose a graph autoencoder framework that first encodes smoothed graph representations with GCN and then decodes accurate graph signals with GDN. We demonstrate the effectiveness of the proposed method on several tasks including unsupervised graph-level representation , social recommendation and graph generation.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[3]  Cesare Alippi,et al.  Spectral Clustering with Graph Neural Networks for Graph Pooling , 2019, ICML.

[4]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.

[5]  Stefano Ermon,et al.  Graphite: Iterative Generative Modeling of Graphs , 2018, ICML.

[6]  Zhiru Zhang,et al.  GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding , 2020, ICLR.

[7]  Muriel Médard,et al.  Network deconvolution as a general method to distinguish direct dependencies in networks , 2013, Nature Biotechnology.

[8]  K. Siddaraju,et al.  DIGITAL IMAGE RESTORATION , 2011 .

[9]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[10]  Subhasis Chaudhuri,et al.  Blind Image Deconvolution , 2014, Springer International Publishing.

[11]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[12]  Yao Zhang,et al.  Sub2Vec: Feature Learning for Subgraphs , 2018, PAKDD.

[13]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[14]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[15]  Matthias Bethge,et al.  A note on the evaluation of generative models , 2015, ICLR.

[16]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[17]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[18]  Yang Liu,et al.  subgraph2vec: Learning Distributed Representations of Rooted Sub-graphs from Large Graphs , 2016, ArXiv.

[19]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[20]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[21]  Jaewoo Kang,et al.  Self-Attention Graph Pooling , 2019, ICML.

[22]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[23]  Pinar Yanardag,et al.  Deep Graph Kernels , 2015, KDD.

[24]  Kurt Mehlhorn,et al.  Efficient graphlet kernels for large graph comparison , 2009, AISTATS.

[25]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[27]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[28]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[30]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[31]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[32]  Ryan G. Coleman,et al.  ZINC: A Free Tool to Discover Chemistry for Biology , 2012, J. Chem. Inf. Model..

[33]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[34]  Xavier Bresson,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[35]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[36]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[37]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[38]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[39]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[40]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[41]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[42]  Jian Tang,et al.  InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization , 2019, ICLR.

[43]  Santiago Segarra,et al.  ENHANCING GEOMETRIC DEEP LEARNING VIA GRAPH FILTER DECONVOLUTION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[44]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[45]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[46]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[47]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[48]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[49]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[51]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[52]  Hong Cheng,et al.  Semi-Supervised Graph Classification: A Hierarchical Graph Perspective , 2019, WWW.

[53]  Honglei Zhang,et al.  Dirichlet Graph Variational Autoencoder , 2020, NeurIPS.

[54]  Risi Kondor,et al.  The Multiscale Laplacian Graph Kernel , 2016, NIPS.