Methods for Optimal Experimental Designs ∗

In this paper, we study optimal experimental design problems with a broad class of smooth convex optimality criteria, including the classical A-, D- and p th mean criterion. In particular, we propose an interior point (IP) method for them and establish its global convergence. Furthermore, by exploiting the structure of the Hessian matrix of the aforementioned optimality criteria, we derive an explicit formula for computing its rank. Using this result, we then show that the Newton direction arising in the IP method can be computed efficiently via Sherman-Morrison-Woodbury formula when the size of the moment matrix is small relative to the sample size. Finally, we compare our IP method with the widely used multiplicative algorithm introduced by Silvey et al. [29]. The computational results show that the IP method generally outperforms the multiplicative algorithm both in speed and solution quality.

[1]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[2]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[3]  Yaming Yu.  Monotonic convergence of a general algorithm for computing optimal designs , 2009, 0905.2646.

[4]  Luc Pronzato,et al.  Improvements on removing nonoptimal support points in D-optimum design algorithms , 2007, 0706.4394.

[5]  Paul Tseng,et al.  Analysis of Nonsmooth Symmetric-Matrix-Valued Functions with Applications to Semidefinite Complementarity Problems , 2003, SIAM J. Optim..

[6]  J. P. Warners,et al.  The Use of Low-rank Updates in Interior-point Methods , 2022 .

[7]  Holger Dette,et al.  Improving updating rules in multiplicative algorithms for computing D-optimal designs , 2008, Comput. Stat. Data Anal..

[8]  Guillaume Sagnol,et al.  Computing Optimal Designs of multiresponse Experiments reduces to Second-Order Cone Programming , 2009, 0912.5467.

[9]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[10]  Margaret H. Wright,et al.  Ill-Conditioning and Computational Error in Interior Methods for Nonlinear Programming , 1998, SIAM J. Optim..

[11]  Margaret H. Wright,et al.  Why a Pure Primal Newton Barrier Step May be Infeasible , 1995, SIAM J. Optim..

[12]  Peng Sun,et al.  Linear convergence of a modified Frank–Wolfe algorithm for computing minimum-volume enclosing ellipsoids , 2008, Optim. Methods Softw..

[13]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[14]  Peter Richtárik.  Simultaneously solving seven optimization problems in relative scale , 2009 .

[15]  B. Torsney.  A Moment Inequality and Monotonicity of an Algorithm , 1983 .

[16]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[17]  S. Nash,et al.  Linear and Nonlinear Optimization , 2008 .

[18]  Radoslav Harman,et al.  Approximate D-optimal designs of experiments on the convex hull of a finite set of information matrices , 2009 .

[19]  Corwin L. Atwood.  Convergent Design Sequences, for Sufficiently Regular Optimality Criteria, II: Singular Case , 1980 .

[20]  Anthony C. Atkinson,et al.  Optimum Experimental Designs, with SAS , 2007 .

[21]  K. Chaloner,et al.  Optimum experimental designs for properties of a compartmental model. , 1993, Biometrics.

[22]  Ben Torsney,et al.  W-Iterations and Ripples Therefrom , 2009 .

[23]  H. Wynn,et al.  The Convergence of General Step-Length Algorithms for Regular Optimum Design Criteria , 1978 .

[24]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[25]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[26]  C. Atwood.  Sequences Converging to $D$-Optimal Designs of Experiments , 1973 .

[27]  Yaming Yu,et al.  D-optimal designs via a cocktail algorithm , 2009, Stat. Comput..

[28]  Luc Pronzato,et al.  Optimal Design and Related Areas in Optimization and Statistics , 2011 .

[29]  Kim-Chuan Toh,et al.  On the Nesterov-Todd Direction in Semidefinite Programming , 1998, SIAM J. Optim..

[30]  Valerii Fedorov,et al.  Design of Experiments in Statistics , 2000 .

[31]  Yong Zhang,et al.  An augmented Lagrangian approach for sparse principal component analysis , 2009, Mathematical Programming.

[32]  R. Martín-Martín,et al.  Multiplicative algorithms for computing optimum designs , 2009 .

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[35]  S. Silvey,et al.  An algorithm for optimal designs on a design space , 1978 .

[36]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[37]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[38]  D'avid Papp,et al.  Optimal Designs for Rational Function Regression , 2010, 1009.1444.

[39]  H. Wynn.  Results in the Theory and Construction of D‐Optimum Experimental Designs , 1972 .

[40]  Dankmar Böhning,et al.  A vertex-exchange-method in D-optimal design theory , 1986 .

[41]  Michael C. Ferris,et al.  Interior-Point Methods for Massive Support Vector Machines , 2002, SIAM J. Optim..

[42]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[43]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[44]  Andrej Pázman,et al.  Foundations of Optimum Experimental Design , 1986 .

[45]  Corwin L. Atwood,et al.  Convergent Design Sequences, for Sufficiently Regular Optimality Criteria , 1976 .

[46]  Radoslav Harman,et al.  Computing c-optimal experimental designs using the simplex method of linear programming , 2008, Comput. Stat. Data Anal..