First-Order Methods for Optimal Experimental Design Problems with Bound Constraints

We consider a class of convex optimization problems over the simplex of probability measures. Our framework comprises optimal experimental design (OED) problems, in which the measure over the design space indicates which experiments are being selected. Due to the presence of additional bound constraints, the measure possesses a Lebesgue density and the problem can be cast as an optimization problem over the space of essentially bounded functions. For this class of problems, we consider two first-order methods including FISTA and a proximal extrapolated gradient method, along with suitable stopping criteria. Finally, acceleration strategies targeting the dimension of the subproblems in each iteration are discussed. Numerical experiments accompany the analysis throughout the paper.

[1]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[2]  Ángel Marín,et al.  Restricted simplicial decomposition with side constraints , 1995, Networks.

[3]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[4]  Maciej Patan,et al.  A sparsity-enforcing method for optimal node activation in parameter estimation of spatiotemporal processes , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[5]  Yaming Yu.  Monotonic convergence of a general algorithm for computing optimal designs , 2009, 0905.2646.

[6]  Maciej Patan,et al.  D-optimal design of a monitoring network for parameter estimation of distributed systems , 2007, J. Glob. Optim..

[7]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[8]  K. Kunisch,et al.  Primal-Dual Strategy for Constrained Optimal Control Problems , 1999 .

[9]  Balder Von Hohenbalken,et al.  Simplicial decomposition in nonlinear programming algorithms , 1977, Math. Program..

[10]  Guillaume Sagnol,et al.  Computing exact D-optimal designs by mixed integer second-order cone programming , 2013, 1307.4953.

[11]  H. P. Wynn,et al.  OPTIMUM SUBMEASURES WITH APPLICATION TO FINITE POPULATION SAMPLING , 1982 .

[12]  G. Minty.  On the monotonicity of the gradient of a convex function. , 1964 .

[13]  Steven E. Rigdon,et al.  Model-Oriented Design of Experiments , 1997, Technometrics.

[14]  Gabriele Steidl,et al.  First order algorithms in variational image processing , 2014, ArXiv.

[15]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[16]  W. Näther.  Optimum experimental designs , 1994 .

[17]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[18]  Valerii V. Fedorov,et al.  Optimal design with bounded density: optimization algorithms of the exchange type , 1989 .

[19]  Yu Malitsky,et al.  Proximal extrapolated gradient methods for variational inequalities , 2016, Optim. Methods Softw..

[20]  Kellen Petersen August.  Real Analysis , 2009 .

[21]  Michael Patriksson.  Simplicial Decomposition Algorithms , 2009, Encyclopedia of Optimization.

[22]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[23]  Anthony C. Atkinson,et al.  Optimum Experimental Designs, with SAS , 2007 .

[24]  Roland Herzog,et al.  Optimal sensor placement for joint parameter and state estimation problems in large-scale dynamical systems with applications to thermo-mechanics , 2018, Optimization and Engineering.

[25]  Ben Torsney,et al.  W-Iterations and Ripples Therefrom , 2009 .

[26]  Boris Vexler,et al.  A sparse control approach to optimal sensor placement in PDE-constrained parameter estimation problems , 2019, Numerische Mathematik.

[27]  J. Kiefer.  General Equivalence Theory for Optimum Designs (Approximate Theory) , 1974 .

[28]  Luc Pronzato,et al.  A minimax equivalence theorem for optimum bounded design measures , 2004 .

[29]  David P. Rutenberg,et al.  Generalized Networks, Generalized Upper Bounding and Decomposition of the Convex Simplex Method , 1970 .

[30]  Dennis Cook,et al.  Constrained Optimization of Experimental Design , 1995 .

[31]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[32]  Roland Herzog,et al.  Optimum Experimental Design for Interface Identification Problems , 2018, SIAM J. Sci. Comput..

[33]  K. I. M. McKinnon,et al.  Solving Stochastic Ship Fleet Routing Problems with Inventory Management Using Branch and Price , 2016, Advances in Stochastic and Deterministic Global Optimization.

[34]  Wotao Yin,et al.  Splitting Methods in Communication, Imaging, Science, and Engineering , 2017 .

[35]  S. Silvey,et al.  An algorithm for optimal designs on a design space , 1978 .

[36]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[37]  H. Wynn.  The Sequential Generation of $D$-Optimum Experimental Designs , 1970 .

[38]  Roland Herzog,et al.  Sequentially optimal sensor placement in thermoelastic models for real time applications , 2015 .

[39]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[40]  Ilya Molchanov,et al.  Optimisation in space of measures and optimal design , 2004 .

[41]  R. Cooke.  Real and Complex Analysis , 2011 .

[42]  Georg Stadler,et al.  Elliptic optimal control problems with L1-control cost and applications for the placement of control devices , 2009, Comput. Optim. Appl..

[43]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .