Verified Global Optimization for Estimating the Parameters of Nonlinear Models

Nonlinear parameter estimation is usually achieved via the minimization of some possibly non-convex cost function. Interval analysis allows one to derive algorithms for the guaranteed characterization of the set of all global minimizers of such a cost function when an explicit expression for the output of the model is available or when this output is obtained via the numerical solution of a set of ordinary differential equations. However, cost functions involved in parameter estimation are usually challenging for interval techniques, if only because of multi-occurrences of the parameters in the formal expression of the cost. This paper addresses parameter estimation via the verified global optimization of quadratic cost functions. It introduces tools for the minimization of generic cost functions. When an explicit expression of the output of the parametric model is available, significant improvements may be obtained by a new box exclusion test and by careful manipulations of the quadratic cost function. When the model is described by ODEs, some of the techniques available in the previous case may still be employed, provided that sensitivity functions of the model output with respect to the parameters are available.

[1]  Hermann Schichl,et al.  Interval Analysis on Directed Acyclic Graphs for Global Optimization , 2005, J. Glob. Optim..

[2]  Woflgang Marquardt,et al.  16th European Symposium on Computer Aided Process Engineering and 9th International Symposium on Process Systems Engineering , 2006 .

[3]  A. Neumaier.  Complete search in continuous global optimization and constraint satisfaction , 2004, Acta Numerica.

[4]  Ralph. Deutsch,et al.  Estimation Theory , 1966 .

[5]  M. Berz,et al.  TAYLOR MODELS AND OTHER VALIDATED FUNCTIONAL INCLUSION METHODS , 2003 .

[6]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[7]  Tibor Csendes,et al.  On the selection of subdivision directions in interval branch-and-bound methods for global optimization , 1995, J. Glob. Optim..

[8]  J. Jacquez.  Compartmental analysis in biology and medicine , 1985 .

[9]  P. Jessup,et al.  A Modern Law of Nations: An Introduction , 1949 .

[10]  Ke Chen,et al.  Applied Mathematics and Computation , 2022 .

[11]  A. Neumaier.  Interval methods for systems of equations , 1990 .

[12]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[13]  Hermann Schichl,et al.  Using directed acyclic graphs to coordinate propagation and search for numerical constraint satisfaction problems , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[14]  L. Goddard,et al.  Operations Research (OR) , 2007 .

[15]  Arnold Neumaier,et al.  Rigorous Enclosures of Ellipsoids and Directed Cholesky Factorizations , 2011, SIAM J. Matrix Anal. Appl..

[16]  Hermann Schichl,et al.  Algorithmic differentiation techniques for global optimization in the COCONUT environment , 2012, Optim. Methods Softw..

[17]  Hermann Schichl,et al.  Exclusion Regions for Systems of Equations , 2004, SIAM J. Numer. Anal..

[18]  Max b. Müller.  Über das Fundamentaltheorem in der Theorie der gewöhnlichen Differentialgleichungen , 1927 .

[19]  Eldon Hansen,et al.  Global optimization using interval analysis , 1992, Pure and applied mathematics.

[20]  Arnold Neumaier,et al.  Constraint propagation on quadratic constraints , 2010, Constraints.

[21]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[22]  M. Stadtherr,et al.  Validated solution of ODEs with parametric uncertainties , 2006 .

[23]  Arnold Neumaier,et al.  The Krawczyk operator and Kantorovich's theorem , 1990 .

[24]  E. Hofer,et al.  VALENCIA-IVP: A Comparison with Other Initial Value Problem Solvers , 2006, 12th GAMM - IMACS International Symposium on Scientific Computing, Computer Arithmetic and Validated Numerics (SCAN 2006).

[25]  R. B. Kearfott.  Rigorous Global Search: Continuous Problems , 1996 .

[26]  Ronald R. Willis,et al.  New Computer Methods for Global Optimization , 1990 .

[27]  M. A. Wolfe,et al.  Interval methods for global optimization , 1996 .

[28]  Ramon E. Moore.  Interval arithmetic and automatic error analysis in digital computing , 1963 .

[29]  Wolfram Luther,et al.  Numerical Software with Result Verification , 2004, Lecture Notes in Computer Science.

[30]  A. Land,et al.  An Automatic Method for Solving Discrete Programming Problems , 1960, 50 Years of Integer Programming.

[31]  Nedialko S. Nedialkov,et al.  A New Perspective on the Wrapping Effect in Interval Methods for Initial Value Problems for Ordinary Differential Equations , 2001, Perspectives on Enclosure Methods.

[32]  Andreas Rauh,et al.  Extensions of ValEncIA‐IVP for reduction of overestimation, for simulation of differential algebraic systems, and for dynamical optimization , 2007 .

[33]  J. Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[34]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[35]  M. Berz,et al.  Efficient high-order methods for ODEs and DAEs , 2000 .

[36]  A. Neumaier.  Acta Numerica 2004: Complete search in continuous global optimization and constraint satisfaction , 2004 .

[37]  H. Schichl.  Global Optimization in the COCONUT Project , 2003, Numerical Software with Result Verification.

[38]  S. Skelboe.  Computation of rational interval functions , 1974 .

[39]  Chenyi Hu,et al.  Algorithm 737: INTLIB—a portable Fortran 77 interval standard-function library , 1994, TOMS.

[40]  Ulrich W. Kulisch,et al.  Perspectives on Enclosure Methods , 2001 .

[41]  E. Walter,et al.  Guaranteed optimization of the parameters of continuous-time knowledge-based models , 2006 .

[42]  Gennadi Vainikko,et al.  Higher order methods , 1993 .