Non-negative least-mean-square algorithm

Dynamic system modeling plays a crucial role in the development of techniques for stationary and nonstationary signal processing. Due to the inherent physical characteristics of systems under investigation, nonnegativity is a desired constraint that can usually be imposed on the parameters to estimate. In this paper, we propose a general method for system identification under nonnegativity constraints. We derive the so-called nonnegative least-mean-square algorithm (NNLMS) based on stochastic gradient descent, and we analyze its convergence. Experiments are conducted to illustrate the performance of this approach and consistency with the analysis.

[1]  J. Yorke,et al.  Chaos: An Introduction to Dynamical Systems , 1997 .

[2]  Sergios Theodoridis,et al.  Adaptive Learning in a World of Projections , 2011, IEEE Signal Processing Magazine.

[3]  Ali H. Sayed,et al.  Adaptive Filters , 2008 .

[4]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[5]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[6]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[7]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[8]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[9]  M. Bertero,et al.  Nonnegative least-squares image deblurring: improved gradient projection approaches , 2010 .

[10]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[11]  J. Minkoff,et al.  Comment on the "Unnecessary assumption of statistical independence between reference signal and filter weights in feedforward adaptive systems" , 2001, IEEE Trans. Signal Process..

[12]  Robert M. May,et al.  Simple mathematical models with very complicated dynamics , 1976, Nature.

[13]  Jean-Marc Vesin,et al.  Stochastic analysis of gradient adaptive identification of nonlinear systems with memory for Gaussian data and noisy input and output measurements , 1999, IEEE Trans. Signal Process..

[14]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[15]  José Carlos M. Bermudez,et al.  An improved statistical analysis of the least mean fourth (LMF) adaptive algorithm , 2003, IEEE Trans. Signal Process..

[16]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[17]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[18]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[19]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[20]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .

[21]  D. Brie,et al.  Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling , 2006, IEEE Transactions on Signal Processing.

[22]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[23]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[24]  H Lantéri,et al.  A general method to devise maximum-likelihood signal restoration multiplicative algorithms with non-negativity constraints , 2001, Signal Process..