Least squares problems with inequality constraints as quadratic constraints

Abstract Linear least squares problems with box constraints are commonly solved to find model parameters within bounds based on physical considerations. Common algorithms include Bounded Variable Least Squares (BVLS) and the Matlab function lsqlin. Here, the goal is to find solutions to ill-posed inverse problems that lie within box constraints. To do this, we formulate the box constraints as quadratic constraints, and solve the corresponding unconstrained regularized least squares problem. Using box constraints as quadratic constraints is an efficient approach because the optimization problem has a closed form solution. The effectiveness of the proposed algorithm is investigated through solving three benchmark problems and one from a hydrological application. Results are compared with solutions found by lsqlin, and the quadratically constrained formulation is solved using the L-curve, maximum a posteriori estimation (MAP), and the χ 2 regularization method. The χ 2 regularization method with quadratic constraints is the most effective method for solving least squares problems with box constraints.

[1]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[2]  David G. Chandler,et al.  Soil moisture states, lateral flow, and streamflow generation in a semi‐arid, snowmelt‐driven catchment , 2005 .

[3]  Fabiana Zama,et al.  An iterative method for linear discrete ill-posed problems with box constraints , 2007 .

[4]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[5]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[6]  Rosemary A. Renaut,et al.  A Newton root-finding algorithm for estimating the regularization parameter for solving ill-conditioned least squares problems , 2009 .

[7]  Arnold Neumaier,et al.  Global Optimization by Multilevel Coordinate Search , 1999, J. Glob. Optim..

[8]  Bert W. Rust,et al.  Constrained Least Squares Interval Estimation , 1985 .

[9]  A. Warrick.  Soil Water Dynamics , 2003 .

[10]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[11]  Jodi Mead,et al.  Parameter estimation: A new approach to weighting a priori information , 2007 .

[12]  Jonas Mockus,et al.  Bayesian Approach to Global Optimization , 1989 .

[13]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[14]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[15]  Jodi Mead.  A priori weighting for parameter estimation , 2008 .

[16]  A. Bennett,et al.  Inverse Modeling of the Ocean and Atmosphere , 2002 .

[17]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[18]  V. Morozov.  On the solution of functional equations by the method of regularization , 1966 .

[19]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[20]  Danny C. Sorensen,et al.  A Trust-Region Approach to the Regularization of Large-Scale Discrete Forms of Ill-Posed Problems , 2001, SIAM J. Sci. Comput..

[21]  Van Genuchten,et al.  A closed-form equation for predicting the hydraulic conductivity of unsaturated soils , 1980 .

[22]  L. A. Richards.  Capillary conduction of liquids through porous mediums , 1931 .

[23]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[24]  D. Menemenlis.  Inverse Modeling of the Ocean and Atmosphere , 2002 .

[25]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[26]  Zbigniew Michalewicz,et al.  GENOCOP: a genetic algorithm for numerical optimization problems with linear constraints , 1996, CACM.

[27]  Per Lötstedt.  Solving the minimal least squares problem subject to bounds on the variables , 1984 .

[28]  Y. Mualem.  A New Model for Predicting the Hydraulic Conductivity , 1976 .

[29]  M. Gribb,et al.  Design of a User‐Friendly Automated Multistep Outflow Apparatus , 2009 .

[30]  P. Stark.  Bounded-Variable Least-Squares: an Algorithm and Applications , 2008 .

[31]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[32]  M. Bierlaire,et al.  On iterative algorithms for linear least squares problems with bound constraints , 1991 .