Discrete Signal Processing on Graphs: Sampling Theory

We propose a sampling theory for signals that are supported on either directed or undirected graphs. The theory follows the same paradigm as classical sampling theory. We show that perfect recovery is possible for graph signals bandlimited under the graph Fourier transform. The sampled signal coefficients form a new graph signal, whose corresponding graph structure preserves the first-order difference of the original graph signal. For general graphs, an optimal sampling operator based on experimentally designed sampling is proposed to guarantee perfect recovery and robustness to noise; for graphs whose graph Fourier transforms are frames with maximal robustness to erasures as well as for Erdös-Rényi graphs, random sampling leads to perfect recovery with high probability. We further establish the connection to the sampling theory of finite discrete-time signal processing and previous work on signal recovery on graphs. To handle full-band graph signals, we propose a graph filter bank based on sampling theory on graphs. Finally, we apply the proposed sampling theory to semi-supervised classification of online blogs and digit images, where we achieve similar or better performance with fewer labeled samples compared to previous work.

[1]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[2]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[3]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[4]  José M. F. Moura,et al.  Signal inpainting on graphs via total variation minimization , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Sunil K. Narang,et al.  Signal processing techniques for interpolation in graph structured data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[7]  Antonio Ortega,et al.  Active semi-supervised learning using sampling theory for graph signals , 2014, KDD.

[8]  Minh N. Do,et al.  Downsampling of Signals on Graphs Via Maximum Spanning Trees , 2015, IEEE Transactions on Signal Processing.

[9]  Antonio Ortega,et al.  Towards a sampling theorem for signals on arbitrary graphs , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[11]  Jelena Kovacevic,et al.  Algebraic Signal Processing Theory: Sampling for Infinite and Finite 1-D Space , 2010, IEEE Transactions on Signal Processing.

[12]  Vivek K Goyal,et al.  Foundations of Signal Processing , 2014 .

[13]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[14]  Pierre Borgnat,et al.  Graph Wavelets for Multiscale Community Mining , 2014, IEEE Transactions on Signal Processing.

[15]  José M. F. Moura,et al.  Adaptive graph filtering: Multiresolution classification on graphs , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[16]  Sunil K. Narang,et al.  Compact Support Biorthogonal Wavelet Filterbanks for Arbitrary Undirected Graphs , 2012, IEEE Transactions on Signal Processing.

[17]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  José M. F. Moura,et al.  Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure , 2014, IEEE Signal Processing Magazine.

[19]  Jelena Kovacevic,et al.  Real, tight frames with maximal robustness to erasures , 2005, Data Compression Conference.

[20]  M. Unser Sampling-50 years after Shannon , 2000, Proceedings of the IEEE.

[21]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[22]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[23]  José M. F. Moura,et al.  Signal Recovery on Graphs: Variation Minimization , 2014, IEEE Transactions on Signal Processing.

[24]  Lada A. Adamic,et al.  The political blogosphere and the 2004 U.S. election: divided they blog , 2005, LinkKDD '05.

[25]  Pascal Frossard,et al.  Clustering on Multi-Layer Graphs via Subspace Analysis on Grassmann Manifolds , 2013, IEEE Transactions on Signal Processing.

[26]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[27]  Jelena Kovacevic,et al.  Signal recovery on graphs: Random versus experimentally designed sampling , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[28]  Nikhil Srivastava,et al.  Graph Sparsification by Effective Resistances , 2011, SIAM J. Comput..

[29]  Yue M. Lu,et al.  A Spectral Graph Uncertainty Principle , 2012, IEEE Transactions on Information Theory.

[30]  I. Pesenson Sampling in paley-wiener spaces on combinatorial graphs , 2008, 1111.5896.

[31]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Foundation and 1-D Time , 2008, IEEE Transactions on Signal Processing.

[32]  James H. Garrett,et al.  Semi-Supervised Multiresolution Classification Using Adaptive Graph Filtering With Application to Indirect Bridge Structural Health Monitoring , 2014, IEEE Transactions on Signal Processing.

[33]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: 1-D Space , 2008, IEEE Transactions on Signal Processing.

[34]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[35]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[36]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[37]  Pascal Frossard,et al.  Learning Parametric Dictionaries for Signals on Graphs , 2014, IEEE Transactions on Signal Processing.

[38]  Martin Vetterli,et al.  A theory of multirate filter banks , 1987, IEEE Trans. Acoust. Speech Signal Process..

[39]  Alfred O. Hero,et al.  Local Fiedler vector centrality for detection of deep and overlapping communities in networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Kannan Ramchandran,et al.  Multiresolution graph signal processing via circulant structures , 2013, 2013 IEEE Digital Signal Processing and Signal Processing Education Meeting (DSP/SPE).

[41]  Leo Grady,et al.  Anisotropic Interpolation on Graphs: The Combinatorial Dirichlet Problem , 2003 .

[42]  José M. F. Moura,et al.  Signal Recovery on Graphs , 2014, ArXiv.

[43]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[44]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[45]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[46]  Jelena Kovacevic,et al.  Systematic Construction of Real Lapped Tight Frame Transforms , 2010, IEEE Transactions on Signal Processing.

[47]  Christos Boutsidis,et al.  Faster Subset Selection for Matrices and Applications , 2011, SIAM J. Matrix Anal. Appl..

[48]  Kannan Ramchandran,et al.  Wavelet-regularized graph semi-supervised learning , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[49]  J. Kovacevic,et al.  Sampling theory for graph signals , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[50]  Pengfei Liu,et al.  Local-Set-Based Graph Signal Reconstruction , 2014, IEEE Transactions on Signal Processing.

[51]  Isaac Z. Pesenson,et al.  Variational Splines and Paley–Wiener Spaces on Combinatorial Graphs , 2009, ArXiv.

[52]  Patrick Flandrin,et al.  Graph Empirical Mode Decomposition , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[53]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[54]  José M. F. Moura,et al.  Signal denoising on graphs via graph filtering , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[55]  Sunil K. Narang,et al.  Unidirectional graph-based wavelet transforms for efficient data gathering in sensor networks , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[56]  Antonio Ortega,et al.  A probabilistic interpretation of sampling theory of graph signals , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Yuantao Gu,et al.  A Distributed Tracking Algorithm for Reconstruction of Graph Signals , 2015, IEEE Journal of Selected Topics in Signal Processing.

[58]  Van H. Vu,et al.  Sparse random graphs: Eigenvalues and eigenvectors , 2010, Random Struct. Algorithms.

[59]  Pascal Frossard,et al.  Learning Graphs from Signal Observations under Smoothness Prior , 2014, ArXiv.

[60]  Isaac Z. Pesenson,et al.  Poincaré and Plancherel-Polya Inequalities in Harmonic Analysis on Weighted Combinatorial Graphs , 2013, SIAM J. Discret. Math..

[61]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[62]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[63]  Jose M. F. Moura,et al.  Representation and processing of massive data sets with irregular structure ] Big Data Analysis with Signal Processing on Graphs , 2022 .

[64]  José M. F. Moura,et al.  Algebraic Signal Processing Theory , 2006, ArXiv.