Transient analysis of l0-LMS and l0-NLMS algorithms

Sparsity-aware adaptive algorithms present some advantages over standard ones, specially due to the fact that they have faster convergence rate. This paper proposes a stochastic model for both l0-LMS and l0-NLMS algorithms, and carries out an accurate transient analysis of these algorithms without requiring the input signal to be white. Some previously unreported learning properties of these algorithms are revealed by the proposed framework, which are confirmed by experimental evidence. HighlightsAn accurate transient analysis of both l0-LMS and l0-NLMS algorithms is advanced.A stochastic model for predicting their MSD and MSE learning behaviors is proposed.The analysis employs weaker conditions, e.g., input signal whiteness is not required.Learning properties of these algorithms are revealed by the proposed framework.Steady-state results can be predicted as limiting cases of the transient analysis.

[1]  Ligang Liu,et al.  An improved mu-law proportionate NLMS algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Georgios B. Giannakis,et al.  Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$ -Norm , 2010, IEEE Transactions on Signal Processing.

[3]  Neil J. Bershad,et al.  On the probability density function of the LMS adaptive filter weights , 1989, IEEE Trans. Acoust. Speech Signal Process..

[4]  Mike Brookes,et al.  Adaptive algorithms for sparse echo cancellation , 2006, Signal Process..

[5]  Yuantao Gu,et al.  $l_{0}$ Norm Constraint LMS Algorithm for Sparse System Identification , 2009, IEEE Signal Processing Letters.

[6]  W. F. Schreiber,et al.  Advanced television systems for terrestrial broadcasting: Some problems and some proposed solutions , 1995, Proc. IEEE.

[7]  K. Senne,et al.  Performance advantage of complex LMS for controlling narrow-band adaptive arrays , 1981 .

[8]  Ali H. Sayed,et al.  Adaptive Filters , 2008 .

[9]  Paulo S. R. Diniz,et al.  Sparsity-Aware Data-Selective Adaptive Filters , 2014, IEEE Transactions on Signal Processing.

[10]  Sergios Theodoridis,et al.  Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted $\ell_{1}$ Balls , 2010, IEEE Transactions on Signal Processing.

[11]  Rui Seara,et al.  Stochastic Model for the NLMS Algorithm with Correlated Gaussian Data , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[12]  Badong Chen,et al.  Mean-Square Convergence Analysis of ADALINE Training With Minimum Error Entropy Criterion , 2010, IEEE Transactions on Neural Networks.

[13]  B. Widrow,et al.  Adaptive antenna systems , 1967 .

[14]  Kevin T. Wagner,et al.  Towards analytical convergence analysis of proportionate-type nlms algorithms , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[15]  Sheng Zhang,et al.  New Steady-State Analysis Results of Variable Step-Size LMS Algorithm With Different Noise Distributions , 2014, IEEE Signal Processing Letters.

[16]  Ali H. Sayed,et al.  On the learning mechanism of adaptive filters , 2000, IEEE Trans. Signal Process..

[17]  S. Thomas Alexander,et al.  Analytical probability density functions for LMS adaptive filters using the Fokker-Planck equation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[18]  Sergios Theodoridis,et al.  A Sparsity Promoting Adaptive Algorithm for Distributed Learning , 2012, IEEE Transactions on Signal Processing.

[19]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[20]  Jian Wang,et al.  Performance Analysis of $l_0$ Norm Constraint Least Mean Square Algorithm , 2012, IEEE Transactions on Signal Processing.

[21]  Diego B. Haddad,et al.  Transient and steady-state MSE analysis of the IMPNLMS algorithm , 2014, Digit. Signal Process..

[22]  Paulo S. R. Diniz,et al.  Affine projection algorithms for sparse system identification , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Yuantao Gu,et al.  Robust zero-point attraction least mean square algorithm on near sparse system identification , 2013, IET Signal Process..

[24]  Paulo S. R. Diniz,et al.  Adaptive Filtering: Algorithms and Practical Implementation , 1997 .

[25]  J. Zeidler,et al.  Second-order output statistics of the adaptive line enhancer , 1979 .

[26]  Claude Samson,et al.  Fixed point error analysis of the normalized ladder algorithm , 1983 .

[27]  R. Seara,et al.  Stochastic Modeling of the Transform-Domain $\varepsilon {\rm LMS}$ Algorithm , 2008, IEEE Transactions on Signal Processing.

[28]  F. Y. Wu,et al.  Non-Uniform Norm Constraint LMS Algorithm for Sparse System Identification , 2013, IEEE Communications Letters.

[29]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[30]  Tareq Y. Al-Naffouri,et al.  Transient analysis of adaptive filters with error nonlinearities , 2003, IEEE Trans. Signal Process..

[31]  R. Seara,et al.  Stochastic analysis of the modified DNLMS algorithm for Gaussian data , 2006, 2006 International Telecommunications Symposium.

[32]  Donald L. Duttweiler,et al.  Proportionate normalized least-mean-squares adaptation in echo cancelers , 2000, IEEE Trans. Speech Audio Process..

[33]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[34]  Armando Manduca,et al.  Highly Undersampled Magnetic Resonance Image Reconstruction via Homotopic $\ell_{0}$ -Minimization , 2009, IEEE Transactions on Medical Imaging.

[35]  Zhaoyang Zhang,et al.  Diffusion Sparse Least-Mean Squares Over Networks , 2012, IEEE Transactions on Signal Processing.

[36]  Ali H. Sayed,et al.  H∞ optimality of the LMS algorithm , 1996, IEEE Trans. Signal Process..

[37]  Tyseer Aboulnasr,et al.  Complexity reduction of the NLMS algorithm via selective coefficient update , 1999, IEEE Trans. Signal Process..

[38]  Iker Sobrón,et al.  Stability and MSE analyses of affine projection algorithms for sparse system identification , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Stefan Werner,et al.  Partial-update NLMS algorithms with data-selective updating , 2004, IEEE Trans. Signal Process..

[40]  Eweda Eweda,et al.  Stochastic Analysis of the LMS and NLMS Algorithms for Cyclostationary White Gaussian Inputs , 2014, IEEE Transactions on Signal Processing.

[41]  Kevin T. Wagner,et al.  Probability Density of Weight Deviations Given Preceding Weight Deviations for Proportionate-Type LMS Adaptive Algorithms , 2011, IEEE Signal Processing Letters.

[42]  Yuantao Gu,et al.  Sparse constraint affine projection algorithm with parallel implementation and application in compressive sensing , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[44]  Isao Yamada,et al.  Sparsity-aware adaptive filters based on ℓp-norm inspired soft-thresholding technique , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[45]  Alfred O. Hero,et al.  Sparse LMS for system identification , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Bijit Kumar Das,et al.  Sparse Adaptive Filtering by an Adaptive Convex Combination of the LMS and the ZA-LMS Algorithms , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[47]  Isao Yamada,et al.  Steady-State Mean-Square Performance Analysis of a Relaxed Set-Membership NLMS Algorithm by the Energy Conservation Argument , 2009, IEEE Transactions on Signal Processing.

[48]  Ali H. Sayed,et al.  LMS and Backpropagation are Minimax Filters , 1994 .

[49]  Yuantao Gu,et al.  A Stochastic Gradient Approach on Compressive Sensing Signal Reconstruction Based on Adaptive Filtering Framework , 2010, IEEE Journal of Selected Topics in Signal Processing.