Deconvolutional Networks on Graph Data

In this paper, we consider an inverse problem in graph learning domain – “given the graph representations smoothed by Graph Convolutional Network (GCN), how can we reconstruct the input graph signal?" We propose Graph Deconvolutional Network (GDN) and motivate the design of GDN via a combination of inverse filters in spectral domain and de-noising layers in wavelet domain, as the inverse operation results in a high frequency amplifier and may amplify the noise. We demonstrate the effectiveness of the proposed method on several tasks including graph feature imputation and graph structure generation.

[1]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[2]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[3]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[4]  Subhasis Chaudhuri,et al.  Blind Image Deconvolution , 2014, Springer International Publishing.

[5]  Nils M. Kriege,et al.  Subgraph Matching Kernels for Attributed Graphs , 2012, ICML.

[6]  Hong Cheng,et al.  Predicting Path Failure In Time-Evolving Graphs , 2019, KDD.

[7]  Santiago Segarra,et al.  ENHANCING GEOMETRIC DEEP LEARNING VIA GRAPH FILTER DECONVOLUTION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[8]  Hamid R. Rabiee,et al.  Deep Graph Generators: A Survey , 2020, IEEE Access.

[9]  Ryan G. Coleman,et al.  ZINC: A Free Tool to Discover Chemistry for Biology , 2012, J. Chem. Inf. Model..

[10]  Jure Leskovec,et al.  Handling Missing Data with Graph Representation Learning , 2020, NeurIPS.

[11]  C. L. Philip Chen,et al.  Graph deconvolutional networks , 2020, Inf. Sci..

[12]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[13]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[14]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[15]  K. Siddaraju,et al.  DIGITAL IMAGE RESTORATION , 2011 .

[16]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[17]  Xavier Bresson,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[18]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Muriel Médard,et al.  Network deconvolution as a general method to distinguish direct dependencies in networks , 2013, Nature Biotechnology.

[20]  Tsuyoshi Murata,et al.  Graph Convolutional Networks for Graphs Containing Missing Features , 2020, Future Gener. Comput. Syst..

[21]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[22]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Qi Liu,et al.  Constrained Graph Variational Autoencoders for Molecule Design , 2018, NeurIPS.

[24]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[25]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[26]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[27]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[28]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[29]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[30]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[31]  Zhiru Zhang,et al.  GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding , 2019, ICLR.

[32]  Ullrich Köthe,et al.  Analyzing Inverse Problems with Invertible Neural Networks , 2018, ICLR.

[33]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[34]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[35]  Stefano Ermon,et al.  Graphite: Iterative Generative Modeling of Graphs , 2018, ICML.

[36]  Jeff Irion,et al.  Efficient Approximation and Denoising of Graph Signals Using the Multiscale Basis Dictionaries , 2017, IEEE Transactions on Signal and Information Processing over Networks.

[37]  Tingyang Xu,et al.  DropEdge: Towards Deep Graph Convolutional Networks on Node Classification , 2020, ICLR.

[38]  Mihaela van der Schaar,et al.  GAIN: Missing Data Imputation using Generative Adversarial Nets , 2018, ICML.

[39]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[40]  Regina Barzilay,et al.  Junction Tree Variational Autoencoder for Molecular Graph Generation , 2018, ICML.

[41]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[42]  Hong Cheng,et al.  Dirichlet Graph Variational Autoencoder , 2020, NeurIPS.

[43]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[44]  Hyung Jin Chang,et al.  Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[45]  Matthias Bethge,et al.  A note on the evaluation of generative models , 2015, ICLR.

[46]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[47]  Stef van Buuren,et al.  MICE: Multivariate Imputation by Chained Equations in R , 2011 .