Enhanced incremental LMS with norm constraints for distributed in-network estimation

This paper addresses the problem of distributed in-network estimation for a vector of interest, which is sparse in nature. To exploit the underlying sparsity of the considered vector, the @?"1 and @?"0 norms are incorporated into the quadratic cost function of the standard distributed incremental least-mean-square (DILMS) algorithm, and some sparse DILMS (Sp-DILMS) algorithms are proposed correspondingly. The performances of the proposed Sp-DILMS algorithms in the mean and mean-square derivation are analyzed. Mathematical analyses show that the Sp-DILMS outperforms the DILMS, if a suitable intensity of the zero-point attractor is selected. Considering that such intensity may not be easily determined in real cases, a new adaptive strategy is designed for its selection. Its effectiveness is verified by both theoretical analysis and numerical simulations. Even though the criterion for intensity selection is derived from the case that the observations are white and Gaussian, simulation results show that it still provides an empirical good choice if the observations are correlated regression vectors.

[1]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Isao Yamada,et al.  Diffusion least-mean squares with adaptive combiners , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Gregory J. Pottie,et al.  Instrumenting the world with wireless sensor networks , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[4]  Yuantao Gu,et al.  $l_{0}$ Norm Constraint LMS Algorithm for Sparse System Identification , 2009, IEEE Signal Processing Letters.

[5]  Mike Brookes,et al.  Adaptive algorithms for sparse echo cancellation , 2006, Signal Process..

[6]  Aníbal R. Figueiras-Vidal,et al.  Adaptively Biasing the Weights of Adaptive Filters , 2010, IEEE Transactions on Signal Processing.

[7]  Jonathon A. Chambers,et al.  A new incremental affine projection-based adaptive algorithm for distributed networks , 2008, Signal Process..

[8]  G. Wahba,et al.  A GENERALIZED APPROXIMATE CROSS VALIDATION FOR SMOOTHING SPLINES WITH NON-GAUSSIAN DATA , 1996 .

[9]  Peng Shi,et al.  Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal , 2010, Signal Process..

[10]  Yingshu Li,et al.  Sparse target counting and localization in sensor networks based on compressive sensing , 2011, 2011 Proceedings IEEE INFOCOM.

[11]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[12]  Milica Stojanovic,et al.  Sparse equalization for real-time digital underwater acoustic communications , 1995, 'Challenges of Our Changing Global Environment'. Conference Proceedings. OCEANS '95 MTS/IEEE.

[13]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[14]  G. Wahba,et al.  A completely automatic french curve: fitting spline functions by cross validation , 1975 .

[15]  Isao Yamada,et al.  Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis , 2010, IEEE Transactions on Signal Processing.

[16]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[17]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[18]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[19]  Ali H. Sayed,et al.  Incremental Adaptive Strategies Over Distributed Networks , 2007, IEEE Transactions on Signal Processing.

[20]  Runze Li,et al.  Tuning parameter selectors for the smoothly clipped absolute deviation method. , 2007, Biometrika.

[21]  Ali H. Sayed,et al.  Fundamentals Of Adaptive Filtering , 2003 .

[22]  Robert D. Nowak,et al.  Compressed channel sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[23]  Ali H. Sayed,et al.  Analysis of Spatial and Incremental LMS Processing for Distributed Estimation , 2011, IEEE Transactions on Signal Processing.

[24]  Ali H. Sayed,et al.  Diffusion Strategies for Distributed Kalman Filtering and Smoothing , 2010, IEEE Transactions on Automatic Control.

[25]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[26]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[27]  A. Hero,et al.  Regularized Least-Mean-Square Algorithms , 2010, 1012.5066.

[28]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[29]  Petre Stoica,et al.  On biased estimators and the unbiased Cramér-Rao lower bound , 1990, Signal Process..

[30]  Zhaoyang Zhang,et al.  Diffusion Sparse Least-Mean Squares Over Networks , 2012, IEEE Transactions on Signal Processing.

[31]  Donald L. Duttweiler,et al.  Proportionate normalized least-mean-squares adaptation in echo cancelers , 2000, IEEE Trans. Speech Audio Process..

[32]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[33]  Ali H. Sayed,et al.  Distributed Adaptive Incremental Strategies: Formulation and Performance Analysis , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[34]  Meta M. Voelker,et al.  Variable Selection and Model Building via Likelihood Basis Pursuit , 2004 .

[35]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.