Discrete Signal Processing on Graphs

In social settings, individuals interact through webs of relationships. Each individual is a node in a complex network (or graph) of interdependencies and generates data, lots of data. We label the data by its source, or formally stated, we index the data by the nodes of the graph. The resulting signals (data indexed by the nodes) are far removed from time or image signals indexed by well ordered time samples or pixels. DSP, discrete signal processing, provides a comprehensive, elegant, and efficient methodology to describe, represent, transform, analyze, process, or synthesize these well ordered time or image signals. This paper extends to signals on graphs DSP and its basic tenets, including filters, convolution, z-transform, impulse response, spectral representation, Fourier transform, frequency response, and illustrates DSP on graphs by classifying blogs, linear predicting and compressing data from irregularly located weather stations, or predicting behavior of customers of a mobile service provider.

[1]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[2]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[3]  Yue M. Lu,et al.  A Spectral Graph Uncertainty Principle , 2012, IEEE Transactions on Information Theory.

[4]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Foundation and 1-D Time , 2008, IEEE Transactions on Signal Processing.

[5]  David G. Stork,et al.  Pattern Classification , 1973 .

[6]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[7]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[8]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[9]  Deborah Estrin,et al.  Multiresolution storage and search in sensor networks , 2005, TOS.

[10]  P. P. Vaidyanathan,et al.  The Theory of Linear Prediction , 2008, Synthesis Lectures on Signal Processing.

[11]  Jelena Kovacevic,et al.  Algebraic Signal Processing Theory: Cooley-Tukey-Type Algorithms for Polynomial Transforms Based on Induction , 2010, SIAM J. Matrix Anal. Appl..

[12]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[13]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: 1-D Space , 2008, IEEE Transactions on Signal Processing.

[14]  Sunil K. Narang,et al.  Local two-channel critically sampled filter-banks on graphs , 2010, 2010 IEEE International Conference on Image Processing.

[15]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[16]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[17]  A. Willsky.  Multiresolution Markov models for signal and image processing , 2002, Proc. IEEE.

[18]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[19]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[20]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[21]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[22]  A.K. Krishnamurthy,et al.  Multidimensional digital signal processing , 1985, Proceedings of the IEEE.

[23]  Gregory Gutin,et al.  Digraphs - theory, algorithms and applications , 2002 .

[24]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: Cooley–Tukey Type Algorithms for DCTs and DSTs , 2007, IEEE Transactions on Signal Processing.

[25]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[26]  Технология,et al.  National Climatic Data Center , 2011 .

[27]  Martin Rötteler,et al.  Algebraic Signal Processing Theory: 2-D Spatial Hexagonal Lattice , 2007, IEEE Trans. Image Process..

[28]  José M. F. Moura,et al.  Discrete signal processing on graphs: Graph filters , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[29]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[30]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[31]  J. Hammersley,et al.  Monte Carlo Methods , 1965 .

[32]  R.G. Baraniuk,et al.  An architecture for distributed wavelet analysis and processing in sensor networks , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[33]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[34]  José M. F. Moura,et al.  Discrete signal processing on graphs: Graph fourier transform , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[35]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[36]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[37]  Patrick J. Wolfe,et al.  Toward signal processing theory for graphs and non-Euclidean data , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[38]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[40]  José M. F. Moura,et al.  Finding Non-Overlapping Clusters for Generalized Inference Over Graphical Models , 2011, IEEE Transactions on Signal Processing.

[41]  Christine Connolly,et al.  Handbook of Image and Video Processing 2nd Edition (Hardback) , 2006 .

[42]  V. Delouille,et al.  Distributed Wavelet De-Noising for Sensor Networks , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[43]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[44]  Graham J. Wills,et al.  Introduction to graphical modelling , 1995 .

[45]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2006, IEEE Transactions on Knowledge and Data Engineering.

[46]  Michael I. Jordan.  Graphical Models , 1998 .

[47]  C. Chamley.  Rational Herds: Economic Models of Social Learning , 2003 .

[48]  C. Guestrin,et al.  Distributed regression: an efficient framework for modeling sensor network data , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[49]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[50]  R. Wagner.  Distributed Wavelet DeNoising for Sensor Networks , 2006 .

[51]  A. Sandryhaila,et al.  Nearest-neighbor image model , 2012, 2012 19th IEEE International Conference on Image Processing.

[52]  B. A. Miller,et al.  Matched filtering for subgraph detection in dynamic networks , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[53]  E. David,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World , 2010 .

[54]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[55]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[56]  Yue M. Lu,et al.  Uncertainty principles for signals defined on graphs: Bounds and characterizations , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Mikhail Belkin,et al.  Using Manifold Stucture for Partially Labeled Classification , 2002, NIPS.

[58]  José M. F. Moura,et al.  Algebraic Signal Processing Theory , 2006, ArXiv.

[59]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[60]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[61]  Lada A. Adamic,et al.  The political blogosphere and the 2004 U.S. election: divided they blog , 2005, LinkKDD '05.

[62]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[63]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[64]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Cooley–Tukey Type Algorithms for Real DFTs , 2008, IEEE Transactions on Signal Processing.

[65]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[66]  Jelena Kovacevic,et al.  Efficient Compression of QRS Complexes Using Hermite Expansion , 2012, IEEE Transactions on Signal Processing.

[67]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[68]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: multiscale methods. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Hyeokho Choi,et al.  Distributed wavelet transform for irregular sensor network grids , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[70]  Jelena Kovacevic,et al.  Algebraic Signal Processing Theory: 1-D Nearest Neighbor Models , 2012, IEEE Transactions on Signal Processing.

[71]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[72]  Mikhail Belkin,et al.  Using manifold structure for partially labelled classification , 2002, NIPS 2002.

[73]  José M. F. Moura,et al.  The Algebraic Approach to the Discrete Cosine and Sine Transforms and Their Fast Algorithms , 2003, SIAM J. Comput..

[74]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[75]  Martin J. Wainwright,et al.  Major Advances and Emerging Developments of Graphical Models [From the Guest Editors] , 2010 .

[76]  Chris Arney,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World (Easley, D. and Kleinberg, J.; 2010) [Book Review] , 2013, IEEE Technology and Society Magazine.