Signal Recovery on Graphs: Variation Minimization

We consider the problem of signal recovery on graphs. Graphs model data with complex structure as signals on a graph. Graph signal recovery recovers one or multiple smooth graph signals from noisy, corrupted, or incomplete measurements. We formulate graph signal recovery as an optimization problem, for which we provide a general solution through the alternating direction methods of multipliers. We show how signal inpainting, matrix completion, robust principal component analysis, and anomaly detection all relate to graph signal recovery and provide corresponding specific solutions and theoretical analysis. We validate the proposed methods on real-world recovery problems, including online blog classification, bridge condition identification, temperature estimation, recommender system for jokes, and expert opinion combination of online blog classification.

[1]  Yue M. Lu,et al.  A Spectral Graph Uncertainty Principle , 2012, IEEE Transactions on Information Theory.

[2]  I. Pesenson.  Sampling in paley-wiener spaces on combinatorial graphs , 2008, 1111.5896.

[3]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Foundation and 1-D Time , 2008, IEEE Transactions on Signal Processing.

[4]  José M. F. Moura,et al.  Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure , 2014, IEEE Signal Processing Magazine.

[5]  Hae Young Noh,et al.  Damage quantification and localization algorithms for indirect SHM of bridges , 2014 .

[6]  Sunil K. Narang,et al.  Compact Support Biorthogonal Wavelet Filterbanks for Arbitrary Undirected Graphs , 2012, IEEE Transactions on Signal Processing.

[7]  Morteza Mardani,et al.  Decentralized Sparsity-Regularized Rank Minimization: Algorithms and Applications , 2012, IEEE Transactions on Signal Processing.

[8]  Pascal Frossard,et al.  Learning of structured graph dictionaries , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[10]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[11]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[13]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[14]  José M. F. Moura,et al.  Signal inpainting on graphs via total variation minimization , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[16]  E. Candès,et al.  Near-ideal model selection by ℓ1 minimization , 2008, 0801.0345.

[17]  Avrim Blum,et al.  Veritas: Combining Expert Opinions without Labeled Data , 2009, Int. J. Artif. Intell. Tools.

[18]  Khadija Iqbal,et al.  An introduction , 1996, Neurobiology of Aging.

[19]  Jelena Kovacevic,et al.  Automated Diagnosis of Otitis Media: Vocabulary and Grammar , 2013, Int. J. Biomed. Imaging.

[20]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[21]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[22]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[23]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[24]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: 1-D Space , 2008, IEEE Transactions on Signal Processing.

[25]  Antonio Ortega,et al.  Towards a sampling theorem for signals on arbitrary graphs , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Avrim Blum,et al.  Veritas: Combining Expert Opinions without Labeled Data , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[27]  Lada A. Adamic,et al.  The political blogosphere and the 2004 U.S. election: divided they blog , 2005, LinkKDD '05.

[28]  Kannan Ramchandran,et al.  Wavelet-regularized graph semi-supervised learning , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[29]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[30]  Jose M. F. Moura,et al.  Representation and processing of massive data sets with irregular structure ] Big Data Analysis with Signal Processing on Graphs , 2022 .

[31]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[32]  José M. F. Moura,et al.  Signal denoising on graphs via graph filtering , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[33]  José M. F. Moura,et al.  Algebraic Signal Processing Theory , 2006, ArXiv.

[34]  Sunil K. Narang,et al.  Unidirectional graph-based wavelet transforms for efficient data gathering in sensor networks , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[37]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[38]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[39]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[40]  James H. Garrett,et al.  Indirect structural health monitoring of a simplified laboratory-scale bridge model , 2014 .

[41]  José M. F. Moura,et al.  Adaptive graph filtering: Multiresolution classification on graphs , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[42]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[43]  Pascal Frossard,et al.  Clustering on Multi-Layer Graphs via Subspace Analysis on Grassmann Manifolds , 2013, IEEE Transactions on Signal Processing.

[44]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[45]  T. Chan,et al.  Variational image inpainting , 2005 .

[46]  Tony F. Chan,et al.  The digital TV filter and nonlinear denoising , 2001, IEEE Trans. Image Process..

[47]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[48]  Pascal Frossard,et al.  Parametric dictionary learning for graph signals , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[49]  José M. F. Moura,et al.  Discrete signal processing on graphs: Graph fourier transform , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[50]  Pengfei Liu,et al.  Local-Set-Based Graph Signal Reconstruction , 2014, IEEE Transactions on Signal Processing.

[51]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[52]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[53]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[54]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[55]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[56]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[57]  Andrea Montanari,et al.  Low-rank matrix completion with noisy observations: A quantitative comparison , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[58]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[59]  James H. Garrett,et al.  Indirect structural health monitoring in bridges: scale experiments , 2012 .

[60]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[61]  José M. F. Moura,et al.  Classification via regularization on graphs , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[62]  Vivek K Goyal,et al.  Foundations of Signal Processing , 2014 .

[63]  James H. Garrett,et al.  Semi-Supervised Multiresolution Classification Using Adaptive Graph Filtering With Application to Indirect Bridge Structural Health Monitoring , 2014, IEEE Transactions on Signal Processing.

[64]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[65]  Abderrahim Elmoataz,et al.  Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing , 2008, IEEE Transactions on Image Processing.

[66]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[67]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[68]  Sunil K. Narang,et al.  Signal processing techniques for interpolation in graph structured data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[69]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[70]  Pengfei Liu,et al.  Coarsening graph signal with spectral invariance , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[71]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[72]  Alfred O. Hero,et al.  Local Fiedler vector centrality for detection of deep and overlapping communities in networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).