Algorithmic differentiation techniques for global optimization in the COCONUT environment

We describe algorithmic differentiation as it can be used in algorithms for global optimization. We focus on the algorithmic differentiation methods implemented in the COCONUT Environment for global nonlinear optimization. The COCONUT Environment represents each factorable optimization problem as a directed acyclic graph (DAG). Various inference modules implemented in this software environment can serve as building blocks for solution algorithms. Many of them use techniques based on various forms of algorithmic differentiation for computing approximations or enclosures of functions or their derivatives. The algorithmic differentiation in the COCONUT Environment not only provides point evaluations but also range enclosures of derivatives up to order 3, as well as slopes up to order 2. Care is taken to ensure that rounding errors are treated correctly. The ranges of the enclosures can be tightened by combining the evaluation routines with constraint propagation. Advantages and pitfalls of this method are also outlined.

[1]  H. Schichl.  Global Optimization in the COCONUT Project , 2003, Numerical Software with Result Verification.

[2]  Hermann Schichl,et al.  Interval Analysis on Directed Acyclic Graphs for Global Optimization , 2005, J. Glob. Optim..

[3]  Chryssostomos Chryssostomidis,et al.  Computer methods for design automation , 1992 .

[4]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[5]  Hermann Schichl,et al.  Using directed acyclic graphs to coordinate propagation and search for numerical constraint satisfaction problems , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[6]  Arnold Neumaier,et al.  Constraint propagation on quadratic constraints , 2010, Constraints.

[7]  Ulrich W. Kulisch,et al.  C++ Toolbox for Verified Computing I: Basic Numerical Problems Theory, Algorithms, and Programs , 1997 .

[8]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[9]  Hermann Schichl,et al.  Exclusion regions for optimization problems , 2014, Journal of Global Optimization.

[10]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[11]  Hermann Schichl,et al.  The optimization test environment , 2014 .

[12]  Siegfried M. Rump.  Expansion and estimation of the range of nonlinear functions , 1996, Math. Comput..

[13]  Christodoulos A. Floudas,et al.  αBB: A global optimization method for general constrained nonconvex problems , 1995, J. Glob. Optim..

[14]  R. Horst.  Deterministic global optimization with partition sets whose feasibility is not known: Application to concave minimization, reverse convex constraints, DC-programming, and Lipschitzian optimization , 1988 .

[15]  Frédéric Goualard,et al.  Revising Hull and Box Consistency , 1999, ICLP.

[16]  Michael J. Maher,et al.  Constraint Logic Programming: A Survey , 1994, J. Log. Program..

[17]  Martin Berz,et al.  Verified Integration of ODEs and Flows Using Differential Algebraic Methods on High-Order Taylor Models , 1998, Reliab. Comput..

[18]  K. Grasse,et al.  A general class of branch-and-bound methods in global optimization with some new approaches for concave minimization , 1986 .

[19]  Pascal Van Hentenryck,et al.  Numerica: A Modeling Language for Global Optimization , 1997, IJCAI.

[20]  Ulrich W. Kulisch,et al.  C++ Toolbox for Verified Computing I , 1995 .

[21]  Hermann Schichl,et al.  GLOPT { A Program for Constrained Global Optimization , 1997 .

[22]  H. Zimmermann.  Towards global optimization 2: L.C.W. DIXON and G.P. SZEGÖ (eds.) North-Holland, Amsterdam, 1978, viii + 364 pages, US $ 44.50, Dfl. 100,-. , 1979 .

[23]  R. Baker Kearfott,et al.  The cluster problem in multivariate global optimization , 1994, J. Glob. Optim..

[24]  Paul I. Barton,et al.  On computational differentiation , 1998 .

[25]  R. Hoprst,et al.  Deterministic global optimization with partition sets whose feasibility is not known: Application to concave minimization, reverse convex constraints, DC-programming, and Lipschitzian optimization , 1988 .

[26]  CarleAlan,et al.  ADIFOR-Generating Derivative Codes from Fortran Programs , 1992 .

[27]  R. Baker Kearfott,et al.  Decomposition of arithmetic expressions to improve the behavior of interval iteration for nonlinear systems , 1991, Computing.

[28]  David A. Kendrick,et al.  GAMS : a user's guide, Release 2.25 , 1992 .

[29]  Hermann Schichl,et al.  Comparison and Automated Selection of Local Optimization Solvers for Interval Global Optimization Methods , 2011, SIAM J. Optim..

[30]  Etienne Huens,et al.  Algorithms for Solving Non-Linear Constrained and Optimization Problems: The State of The Art , 2001 .

[31]  Arnold Neumaier,et al.  The Krawczyk operator and Kantorovich's theorem , 1990 .

[32]  R. B. Kearfott.  Rigorous Global Search: Continuous Problems , 1996 .

[33]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[34]  A. Neumaier,et al.  Interval Slopes for Rational Functions and Associated Centered Forms , 1985 .

[35]  Lubomir V. Kolev.  Use of Interval Slopes for the Irrational Part of Factorable Functions , 1997, Reliab. Comput..

[36]  A. Neumaier.  Interval methods for systems of equations , 1990 .

[37]  Arnold Neumaier,et al.  Taylor Forms—Use and Limits , 2003, Reliab. Comput..

[38]  L. Dixon,et al.  Automatic differentiation of algorithms , 2000 .

[39]  Nikolaos V. Sahinidis,et al.  BARON: A general purpose global optimization software package , 1996, J. Glob. Optim..

[40]  Frédéric Goualard,et al.  Box consistency through weak box consistency , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[41]  Eric Walter,et al.  Verified Global Optimization for Estimating the Parameters of Nonlinear Models , 2011 .

[42]  Frédéric Benhamou,et al.  Applying Interval Arithmetic to Real, Integer, and Boolean Constraints , 1997, J. Log. Program..

[43]  Ferenc Domes,et al.  Rigorous techniques for continuous constraint satisfaction problems , 2010 .

[44]  Arnold Neumaier,et al.  Rigorous Enclosures of Ellipsoids and Directed Cholesky Factorizations , 2011, SIAM J. Matrix Anal. Appl..

[45]  C. Bischof,et al.  Structured second-and higher-order derivatives through univariate Taylor series , 1993 .

[46]  Nikolaos V. Sahinidis,et al.  Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming , 2002 .

[47]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[48]  O. Knüppel,et al.  PROFIL/BIAS—A fast interval library , 1994, Computing.

[49]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[50]  Hermann Schichl,et al.  Exclusion Regions for Systems of Equations , 2004, SIAM J. Numer. Anal..

[51]  Boi Faltings,et al.  Consistency techniques for continuous constraints , 1996, Constraints.

[52]  Brian W. Kernighan,et al.  AMPL: a mathematical programming language , 1989 .

[53]  Rajeev Motwani,et al.  Profile-driven instruction level parallel scheduling with application to super blocks , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.

[54]  Alex Pothen,et al.  What Color Is Your Jacobian? Graph Coloring for Computing Derivatives , 2005, SIAM Rev..