Distributed Adaptive Learning of Graph Signals

The aim of this paper is to propose distributed strategies for adaptive learning of signals defined over graphs. Assuming the graph signal to be bandlimited, the method enables distributed reconstruction, with guaranteed performance in terms of mean-square error, and tracking from a limited number of sampled observations taken from a subset of vertices. A detailed mean-square analysis is carried out and illustrates the role played by the sampling strategy on the performance of the proposed method. Finally, some useful strategies for distributed selection of the sampling set are provided. Several numerical results validate our theoretical findings, and illustrate the performance of the proposed method for distributed adaptive learning of signals defined over graphs.

[1]  José M. F. Moura,et al.  Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure , 2014, IEEE Signal Processing Magazine.

[2]  Jelena Kovacevic,et al.  Signal Recovery on Graphs: Fundamental Limits of Sampling Strategies , 2015, IEEE Transactions on Signal and Information Processing over Networks.

[3]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[4]  Cássio Guimarães Lopes,et al.  Towards spatially universal adaptive diffusion networks , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[5]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[6]  Sergios Theodoridis,et al.  Adaptive Robust Distributed Learning in Diffusion Sensor Networks , 2011, IEEE Transactions on Signal Processing.

[7]  José M. F. Moura,et al.  Signal Recovery on Graphs: Variation Minimization , 2014, IEEE Transactions on Signal Processing.

[8]  Ali H. Sayed,et al.  Sparse Distributed Learning Based on Diffusion Adaptation , 2012, IEEE Transactions on Signal Processing.

[9]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[10]  Sergio Barbarossa,et al.  Adaptive Least Mean Squares Estimation of Graph Signals , 2016, IEEE Transactions on Signal and Information Processing over Networks.

[11]  Haris Vikalo,et al.  Greedy sensor selection: Leveraging submodularity , 2010, 49th IEEE Conference on Decision and Control (CDC).

[12]  Ali Sayed,et al.  Adaptation, Learning, and Optimization over Networks , 2014, Found. Trends Mach. Learn..

[13]  Stephen P. Boyd,et al.  Distributed average consensus with least-mean-square deviation , 2007, J. Parallel Distributed Comput..

[14]  Jie Chen,et al.  Multitask Diffusion Adaptation Over Networks , 2013, IEEE Transactions on Signal Processing.

[15]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[16]  Sergio Barbarossa,et al.  Signals on Graphs: Uncertainty Principle and Sampling , 2015, IEEE Transactions on Signal Processing.

[17]  Sunil K. Narang,et al.  Compact Support Biorthogonal Wavelet Filterbanks for Arbitrary Undirected Graphs , 2012, IEEE Transactions on Signal Processing.

[18]  I. Pesenson Sampling in paley-wiener spaces on combinatorial graphs , 2008, 1111.5896.

[19]  Markus Püschel,et al.  Algebraic Signal Processing Theory: Foundation and 1-D Time , 2008, IEEE Transactions on Signal Processing.

[20]  Santiago Segarra,et al.  Sampling of Graph Signals With Successive Local Aggregations , 2015, IEEE Transactions on Signal Processing.

[21]  Sergio Barbarossa,et al.  On the degrees of freedom of signals on graphs , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[22]  José M. F. Moura,et al.  Algebraic Signal Processing Theory: 1-D Space , 2008, IEEE Transactions on Signal Processing.

[23]  Jelena Kovacevic,et al.  Distributed algorithm for graph signal inpainting , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[25]  Ali H. Sayed,et al.  Distributed Spectrum Estimation for Small Cell Networks Based on Sparse Diffusion Adaptation , 2013, IEEE Signal Processing Letters.

[26]  José M. F. Moura,et al.  Signal denoising on graphs via graph filtering , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[27]  Santiago Segarra,et al.  Reconstruction of Graph Signals Through Percolation from Seeding Nodes , 2015, IEEE Transactions on Signal Processing.

[28]  Antonio Ortega,et al.  Active semi-supervised learning using sampling theory for graph signals , 2014, KDD.

[29]  Jie Chen,et al.  Diffusion LMS Over Multitask Networks , 2014, IEEE Transactions on Signal Processing.

[30]  Sunil K. Narang,et al.  Perfect Reconstruction Two-Channel Wavelet Filter Banks for Graph Structured Data , 2011, IEEE Transactions on Signal Processing.

[31]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[32]  J. Silvester Determinants of block matrices , 2000, The Mathematical Gazette.

[33]  BoydStephen,et al.  Distributed average consensus with least-mean-square deviation , 2007 .

[34]  Sergio Barbarossa,et al.  Distributed Estimation and Control of Algebraic Connectivity Over Random Graphs , 2013, IEEE Transactions on Signal Processing.

[35]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[36]  Sunil K. Narang,et al.  Localized iterative methods for interpolation in graph structured data , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[37]  Yuantao Gu,et al.  A Distributed Tracking Algorithm for Reconstruction of Graph Signals , 2015, IEEE Journal of Selected Topics in Signal Processing.

[38]  José M. F. Moura,et al.  Classification via regularization on graphs , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[39]  S. Haykin Adaptive Filters , 2007 .

[40]  José M. F. Moura,et al.  Signal inpainting on graphs via total variation minimization , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  Sundeep Prabhakar Chepuri,et al.  Sparsity-Promoting Sensor Selection for Non-Linear Measurement Models , 2013, IEEE Transactions on Signal Processing.

[42]  Paolo Di Lorenzo,et al.  Diffusion Adaptation Strategies for Distributed Estimation Over Gaussian Markov Random Fields , 2014, IEEE Transactions on Signal Processing.

[43]  Isao Yamada,et al.  Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis , 2010, IEEE Transactions on Signal Processing.

[44]  Pascal Frossard,et al.  Distributed signal processing with graph spectral dictionaries , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[45]  Marc Moonen,et al.  Seeing the Bigger Picture: How Nodes Can Learn Their Place Within a Complex Ad Hoc Network Topology , 2013, IEEE Signal Processing Magazine.

[46]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  Pengfei Liu,et al.  Local-Set-Based Graph Signal Reconstruction , 2014, IEEE Transactions on Signal Processing.

[48]  Jie Chen,et al.  Multitask Diffusion Adaptation Over Networks With Common Latent Representations , 2017, IEEE Journal of Selected Topics in Signal Processing.

[49]  Pascal Frossard,et al.  Parametric dictionary learning for graph signals , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[50]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[51]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[52]  David Kempe,et al.  A decentralized algorithm for spectral analysis , 2004, STOC '04.

[53]  Jelena Kovacevic,et al.  Discrete Signal Processing on Graphs: Sampling Theory , 2015, IEEE Transactions on Signal Processing.

[54]  Ali H. Sayed,et al.  Bio-Inspired Decentralized Radio Access Based on Swarming Mechanisms Over Adaptive Networks , 2013, IEEE Transactions on Signal Processing.

[55]  Sunil K. Narang,et al.  Signal processing techniques for interpolation in graph structured data , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[56]  Sundeep Prabhakar Chepuri,et al.  Subsampling for graph power spectrum estimation , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[57]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[58]  Magno T. M. Silva,et al.  A novel scheme for diffusion networks with least-squares adaptive combiners , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[59]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[60]  Sergio Barbarossa,et al.  Distributed Detection and Estimation in Wireless Sensor Networks , 2013, ArXiv.