A nonlinear preconditioner for optimum experimental design problems

We show how to efficiently compute A-optimal experimental designs, which are formulated in terms of the minimization of the trace of the covariance matrix of the underlying regression process, using quasi-Newton sequential quadratic programming methods. In particular, we introduce a modification of the problem that leads to significantly faster convergence. To derive this modification, we model each iteration in terms of an initial experimental design that is to be improved, and show that the absolute condition number of the model problem grows without bounds as the quality of the initial design improves. As a remedy, we devise a preconditioner that ensures that the absolute condition number will instead stay uniformly bounded. Using numerical experiments, we study the effect of this reformulation on relevant cases of the general problem class, and find that it leads to significant improvements in both stability and convergence behavior.

[1]  Stefan Körkel,et al.  Numerische Methoden für Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen , 2002 .

[2]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[3]  Sandro Macchietto,et al.  Model-based design of experiments for parameter precision: State of the art , 2008 .

[4]  Hans Bock,et al.  Numerical methods for optimum experimental design in DAE systems , 2000 .

[5]  R. FitzHugh Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[6]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[7]  E. Hairer,et al.  Stiff differential equations solved by Radau methods , 1999 .

[8]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[9]  Shih-Ping Han A globally convergent method for nonlinear programming , 1975 .

[10]  David E. Keyes,et al.  Nonlinearly Preconditioned Inexact Newton Algorithms , 2002, SIAM J. Sci. Comput..

[11]  F. Pukelsheim,et al.  Efficient rounding of approximate designs , 1992 .

[12]  K. Schittkowski The nonlinear programming method of Wilson, Han, and Powell with an augmented Lagrangian type line search function , 1982 .

[13]  T. Zolezzi On the Distance Theorem in Quadratic Optimization , 2002 .

[14]  J. Demmel On condition numbers and the distance to the nearest ill-posed problem , 2015 .

[15]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[16]  H. Bock,et al.  Differentiable evaluation of objective functions in sampling design with variance‐covariance matrices , 2011 .

[17]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[18]  X. Yi On Automatic Differentiation , 2005 .

[19]  F. Pukelsheim Optimal Design of Experiments (Classics in Applied Mathematics) (Classics in Applied Mathematics, 50) , 2006 .

[20]  Luc Pronzato,et al.  Optimal experimental design and some related control problems , 2008, Autom..

[21]  Philipp Birken,et al.  INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN FLUIDS Int. J. Numer. Meth. Fluids (2009) Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/fld.2030 On nonlinear preconditioners in Newton–Krylov , 2022 .

[22]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[23]  Philip E. Gill,et al.  User's guide for SOL/QPSOL: a Fortran package for quadratic programming , 1983 .

[24]  Xiaoye S. Li,et al.  Algorithms for quad-double precision floating point arithmetic , 2000, Proceedings 15th IEEE Symposium on Computer Arithmetic. ARITH-15 2001.

[25]  H. Bock Numerical Treatment of Inverse Problems in Chemical Reaction Kinetics , 1981 .

[26]  Hans Bock,et al.  Numerical methods for parameter estimation and optimal experiment design in chemical reaction systems , 1992 .

[27]  F. Pukelsheim Optimal Design of Experiments , 1993 .

[28]  Tullio Zolezzi Condition Number Theorems in Optimization , 2003, SIAM J. Optim..