Distributed Sparse Total Least-Squares Over Networks

Distributed estimation is attracting more and more attention due to its scalability, robustness, and low power-consumption. In most distributed estimation algorithms, the output of the system is assumed to be noisy, while the input data is assumed to be accurate. However, in real applications, both of the input and output data may be perturbed by noise. Thus, it is unrealistic to assume that all the entries in the input data are accurate and only those in the output data are corrupted. In the cases of noisy input and output data, the total least-squares (TLS) method has the ability of minimizing the perturbations in both input and output data, and thus provides a better performance than the least-squares (LS)-based method. Besides, many nature and manmade systems present high level of sparsity. In this paper, we consider the case in which both the input and output data are corrupted by noise, and the parameter of interest is sparse. We present several sparsity-aware distributed TLS algorithms for the in-network cooperative estimation problem, in which the l1- or l0-norm penalty term is used to exploit the sparsity of the signal. We then present theoretical analysis on the mean and mean-square performance of the proposed algorithms. In addition, several numerical simulations are given to verify the effectiveness and advantages of these proposed algorithms.

[1]  Isao Yamada,et al.  Diffusion least-mean squares with adaptive combiners , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Yuantao Gu,et al.  $l_{0}$ Norm Constraint LMS Algorithm for Sparse System Identification , 2009, IEEE Signal Processing Letters.

[3]  Chunguang Li,et al.  Distributed Sparse Recursive Least-Squares Over Networks , 2014, IEEE Transactions on Signal Processing.

[4]  Georgios B. Giannakis,et al.  Distributed cognitive spectrum sensing via group sparse total least-squares , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[5]  Heyu Wang,et al.  Distributed Frequency Estimation Over Sensor Network , 2015, IEEE Sensors Journal.

[6]  H.C. Papadopoulos,et al.  Locally constructed algorithms for distributed computations in ad-hoc networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[7]  Chunguang Li,et al.  Distributed Information Theoretic Clustering , 2014, IEEE Transactions on Signal Processing.

[8]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[9]  Ali H. Sayed,et al.  Incremental Adaptive Strategies Over Distributed Networks , 2007, IEEE Transactions on Signal Processing.

[10]  Tülay Adali,et al.  Unbiased Recursive Least-Squares Estimation Utilizing Dichotomous Coordinate-Descent Iterations , 2014, IEEE Transactions on Signal Processing.

[11]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[12]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[13]  Zhaoyang Zhang,et al.  Diffusion Information Theoretic Learning for Distributed Estimation Over Network , 2013, IEEE Transactions on Signal Processing.

[14]  Geoffrey A. Williamson,et al.  Analysis of gradient algorithms for TLS-based adaptive IIR filters , 2004, IEEE Transactions on Signal Processing.

[15]  M. Swamy,et al.  A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering , 1994 .

[16]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[18]  Azam Khalili,et al.  Steady-State Analysis of Incremental LMS Adaptive Networks With Noisy Links , 2011, IEEE Transactions on Signal Processing.

[19]  Zhaoyang Zhang,et al.  Distributed estimation over complex networks , 2012, Inf. Sci..

[20]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[21]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Peter Bühlmann Regression shrinkage and selection via the Lasso: a retrospective (Robert Tibshirani): Comments on the presentation , 2011 .

[23]  Roberto López-Valcarce,et al.  Distributed Total Least Squares estimation over networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Marc Moonen,et al.  Consensus-Based Distributed Total Least Squares Estimation in Ad Hoc Wireless Sensor Networks , 2011, IEEE Transactions on Signal Processing.

[25]  Georgios B. Giannakis,et al.  Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling , 2010, IEEE Transactions on Signal Processing.

[26]  Sabine Van Huffel,et al.  Overview of total least-squares methods , 2007, Signal Process..

[27]  Stefan Werner,et al.  Diffusion-based distributed adaptive estimation utilizing gradient-descent total least-squares , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Jiasong Wu,et al.  Error Gradient-based Variable-Lp Norm Constraint LMS Algorithm for Sparse System Identification , 2015, ArXiv.

[29]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[30]  Ali H. Sayed,et al.  Sparse Distributed Learning Based on Diffusion Adaptation , 2012, IEEE Transactions on Signal Processing.

[31]  Zhaoyang Zhang,et al.  Diffusion Sparse Least-Mean Squares Over Networks , 2012, IEEE Transactions on Signal Processing.

[32]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[33]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[34]  C. Richard Johnson,et al.  Exploiting sparsity in adaptive filters , 2002, IEEE Trans. Signal Process..

[35]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks With Imperfect Communication: Link Failures and Channel Noise , 2007, IEEE Transactions on Signal Processing.

[36]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[37]  Stefan Werner,et al.  Analysis of the Gradient-Descent Total Least-Squares Adaptive Filtering Algorithm , 2014, IEEE Transactions on Signal Processing.

[38]  Wei Xing Zheng,et al.  A fast recursive total least squares algorithm for adaptive FIR filtering , 2004, IEEE Trans. Signal Process..

[39]  Ali H. Sayed,et al.  Analysis of Spatial and Incremental LMS Processing for Distributed Estimation , 2011, IEEE Transactions on Signal Processing.

[40]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[41]  Azam Khalili,et al.  Steady-State Analysis of Diffusion LMS Adaptive Networks With Noisy Links , 2012, IEEE Transactions on Signal Processing.

[42]  K. J. Ray Liu,et al.  Distributed Adaptive Learning Mechanisms , 2009 .

[43]  Ioannis D. Schizas,et al.  Distributed LMS for Consensus-Based In-Network Adaptive Processing , 2009, IEEE Transactions on Signal Processing.

[44]  Soummya Kar,et al.  Gossip Algorithms for Distributed Signal Processing , 2010, Proceedings of the IEEE.

[45]  Marc Moonen,et al.  Low-Complexity Distributed Total Least Squares Estimation in Ad Hoc Sensor Networks , 2012, IEEE Transactions on Signal Processing.

[46]  Isao Yamada,et al.  Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis , 2010, IEEE Transactions on Signal Processing.