A nonlinear preconditioner for experimental design problems

We address the slow convergence and poor stability of quasi-newton sequential quadratic programming (SQP) methods that is observed when solving experimental design problems, in particular when they are large. Our findings suggest that this behavior is due to the fact that these problems often have bad absolute condition numbers. To shed light onto the structure of the problem close to the solution, we formulate a model problem (based on the $A$-criterion), that is defined in terms of a given initial design that is to be improved. We prove that the absolute condition number of the model problem grows without bounds as the quality of the initial design improves. Additionally, we devise a preconditioner that ensures that the condition number will instead stay uniformly bounded. Using numerical experiments, we study the effect of this reformulation on relevant cases of the general problem, and find that it leads to significant improvements in stability and convergence behavior.

[1]  H. Bock.  Numerical Treatment of Inverse Problems in Chemical Reaction Kinetics , 1981 .

[2]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[3]  J. Demmel.  On condition numbers and the distance to the nearest ill-posed problem , 2015 .

[4]  H. Bock,et al.  Differentiable evaluation of objective functions in sampling design with variance‐covariance matrices , 2011 .

[5]  F. Pukelsheim,et al.  Efficient rounding of approximate designs , 1992 .

[6]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[7]  Sebastian Sager,et al.  Sampling Decisions in Optimum Experimental Design in the Light of Pontryagin's Maximum Principle , 2013, SIAM J. Control. Optim..

[8]  G. Dahlquist.  A special stability problem for linear multistep methods , 1963 .

[9]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[10]  J. NAGUMOt,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 2006 .

[11]  Tullio Zolezzi.  Condition Number Theorems in Optimization , 2003, SIAM J. Optim..

[12]  Hans Bock,et al.  Numerical methods for parameter estimation and optimal experiment design in chemical reaction systems , 1992 .

[13]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[14]  T. Zolezzi.  On the Distance Theorem in Quadratic Optimization , 2002 .

[15]  E. Hairer,et al.  Stiff differential equations solved by Radau methods , 1999 .

[16]  Griewank,et al.  On automatic differentiation , 1988 .

[17]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[18]  Xiaoye S. Li,et al.  Algorithms for quad-double precision floating point arithmetic , 2000, Proceedings 15th IEEE Symposium on Computer Arithmetic. ARITH-15 2001.

[19]  K. Schittkowski.  The nonlinear programming method of Wilson, Han, and Powell with an augmented Lagrangian type line search function , 1982 .

[20]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[21]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[22]  Shih-Ping Han.  A globally convergent method for nonlinear programming , 1975 .

[23]  Stefan Körkel,et al.  Numerische Methoden für Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen , 2002 .

[24]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[25]  Hans Bock,et al.  Numerical methods for optimum experimental design in DAE systems , 2000 .

[26]  R. Fateman.  Building Algebra Systems by Overloading Lisp : Automatic Differentiation , 2006 .

[27]  Sandro Macchietto,et al.  Model-based design of experiments for parameter precision: State of the art , 2008 .