Optimization of Algorithmic Parameters using a Meta-Control Approach*

Optimization algorithms usually rely on the setting of parameters, such as barrier coefficients. We have developed a generic meta-control procedure to optimize the behavior of given iterative optimization algorithms. In this procedure, an optimal continuous control problem is defined to compute the parameters of an iterative algorithm as control variables to achieve a desired behavior of the algorithm (e.g., convergence time, memory resources, and quality of solution). The procedure is illustrated with an interior point algorithm to control barrier coefficients for constrained nonlinear optimization. Three numerical examples are included to demonstrate the enhanced performance of this method.

[1]  W. D. Evans,et al.  PARTIAL DIFFERENTIAL EQUATIONS , 1941 .

[2]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[3]  Michael Athans,et al.  Optimal Control , 1966 .

[4]  Huibert Kwakernaak,et al.  Linear Optimal Control Systems , 1972 .

[5]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[6]  Klaus Schittkowski,et al.  More test examples for nonlinear programming codes , 1981 .

[7]  G. McCormick.  Nonlinear Programming: Theory, Algorithms and Applications , 1983 .

[8]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[9]  P. Olver.  Applications of Lie Groups to Differential Equations , 1986 .

[10]  H. Sherali,et al.  An assumption‐free convergence analysis for a perturbation of the scaling algorithm for linear programs, with application to the L 1 estimation problem , 1988, Naval Research Logistics (NRL).

[11]  R. C. Monteiro,et al.  Interior path following primal-dual algorithms , 1988 .

[12]  Renato D. C. Monteiro,et al.  Interior path following primal-dual algorithms. part II: Convex quadratic programming , 1989, Math. Program..

[13]  Renato D. C. Monteiro,et al.  Interior path following primal-dual algorithms. part I: Linear programming , 1989, Math. Program..

[14]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[15]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[16]  Anil Nerode,et al.  Hybrid Systems: Chattering Approximation to Relaxed Controls , 1995, Hybrid Systems.

[17]  Claes Johnson,et al.  Computational Differential Equations , 1996 .

[18]  A. Nerode,et al.  Continualization : A hybrid systems control technique for computing , 1996 .

[19]  T. Frankel.  The Geometry of Physics , 1997 .

[20]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[21]  J. Butcher.  Numerical methods for ordinary differential equations in the 20th century , 2000 .

[22]  A. Nerode,et al.  CONTROL SYNTHESIS IN HYBRID SYSTEMS WITH FINSLER DYNAMICS , 2002 .

[23]  Anders Forsgren,et al.  Interior Methods for Nonlinear Optimization , 2002, SIAM Rev..

[24]  Paul Tseng,et al.  On some interior-point algorithms for nonconvex quadratic optimization , 2002, Math. Program..

[25]  J. Butcher.  Numerical methods for ordinary differential equations , 2003 .

[26]  J. Butcher.  Numerical Methods for Ordinary Differential Equations: Butcher/Numerical Methods , 2005 .