TNT-NN: A Fast Active Set Method for Solving Large Non-Negative Least Squares Problems

Abstract In 1974 Lawson and Hanson produced a seminal active set strategy to solve least-squares problems with non-negativity constraints that remains popular today. In this paper we present TNT-NN, a new active set method for solving non-negative least squares (NNLS) problems. TNT-NN uses a different strategy not only for the construction of the active set but also for the solution of the unconstrained least squares sub-problem. This results in dramatically improved performance over traditional active set NNLS solvers, including the Lawson and Hanson NNLS algorithm and the Fast NNLS (FNNLS) algorithm, allowing for computational investigations of new types of scientific and engineering problems. For the small systems tested (5000 × 5000 or smaller), it is shown that TNT-NN is up to 95 × faster than FNNLS. Recent studies in rock magnetism have revealed a need for fast NNLS algorithms to address large problems (on the order of 10 5 × 10 5 or larger). We apply the TNT-NN algorithm to a representative rock magnetism inversion problem where it is 60× faster than FNNLS. We also show that TNT-NN is capable of solving large (45000 × 45000) problems more than 150 × faster than FNNLS. These large test problems were previously considered to be unsolvable, due to the excessive execution time required by traditional methods.

[1]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[2]  E. A. Lima,et al.  Paleointensity of the Earth's magnetic field using SQUID microscopy , 2007 .

[3]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[4]  A. L. Albee,et al.  Constrained least-squares analysis of petrologic problems with an application to lunar sample 12040 , 1973 .

[5]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[6]  Yuji Yagi,et al.  Source rupture process of the 2003 Tokachi-oki earthquake determined by joint inversion of teleseismic body wave and strong ground motion data , 2004 .

[7]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[8]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[9]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[10]  David P Gallegos,et al.  A NMR technique for the analysis of pore structure: Determination of continuous pore size distributions , 1988 .

[11]  E. A. Lima,et al.  Paleomagnetic analysis using SQUID microscopy , 2007 .

[12]  Ramani Duraiswami,et al.  Efficient Parallel Nonnegative Least Squares on Multicore Architectures , 2011, SIAM J. Sci. Comput..

[13]  M. Heath.  Numerical Methods for Large Sparse Linear Least Squares Problems , 1984 .

[14]  Laurent Baratchart,et al.  Fast inversion of magnetic field maps of unidirectional planar geological magnetization , 2013 .

[15]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[16]  Karen H. Haskell,et al.  An algorithm for linear least squares problems with equality and nonnegativity constraints , 1981, Math. Program..

[17]  R. Parker.  Geophysical Inverse Theory , 1994 .

[18]  Inderjit S. Dhillon,et al.  Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach , 2010, SIAM J. Sci. Comput..

[19]  David J. Lilja,et al.  TNT: A Solver for Large Dense Least-Squares Problems that Takes Conjugate Gradient from Bad in Theory, to Good in Practice , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[20]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[21]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[22]  Zonghou Xiong,et al.  Three-dimensional earth conductivity inversion , 1992 .

[23]  E. A. Lima,et al.  Unraveling the simultaneous shock magnetization and demagnetization of rocks , 2010 .

[24]  F. Albarède,et al.  Petrological and geochemical mass-balance equations: an algorithm for least-square fitting and general error analysis , 1977 .

[25]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .