A New Projected Quasi-Newton Approach for the Nonnegative Least Squares Problem

Constrained least squares estimation lies at the heart of many applications in fields as diverse as statistics, psychometrics, signal processing, or even machine learning. Nonnegativity requirements on the model variables are amongst the simplest constraints that arise naturally, and the corresponding least-squares problem is called Nonnegative Least Squares or NNLS. In this paper we present a new, efficient, and scalable Quasi-Newton-type method for solving the NNLS problem, improving on several previous approaches and leading to a superlinearly convergent method. We show experimental results comparing our method to well-known methods for solving the NNLS problem. Our method significantly outperforms other methods, especially as the problem size becomes larger.

[1]  J. B. Rosen.  The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints , 1960 .

[2]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[3]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[4]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[5]  J. Dunn.  Global and Asymptotic Convergence Rate Estimates for a Class of Projected Gradient Processes , 1981 .

[6]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, CDC 1981.

[7]  Philip E. Gill,et al.  Practical optimization , 1981 .

[8]  Klaus Schittkowski,et al.  The Numerical Solution of Constrained Linear Least-squares Problems , 1983 .

[9]  李幼升,et al.  Ph , 1989 .

[10]  M. Bierlaire,et al.  On iterative algorithms for linear least squares problems with bound constraints , 1991 .

[11]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[12]  G. Toraldo,et al.  On the identification property of a projected gradient method , 1993 .

[13]  L. N. Vicente,et al.  A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables , 1994 .

[14]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[15]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[16]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[17]  Michael Piatek,et al.  Tsnnls: A solver for large sparse least squares problems with non-negative variables , 2004, ArXiv.

[18]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .