A General-Purpose Software Framework for Dynamic Optimization (Een algemene softwareomgeving voor dynamische optimalisatie)

Methods and software for derivative-based numerical optimization in general and simulation-based optimization in particular have seen a large rise in popularity over the past 30 years. Still, due to practical difficulties in implementing many of the methods in a fast and reliable manner, it remains an underused technology both in academia and in industry. To address this, we present a set of methods and tools with the aim of making techniques for dynamic optimization more accessible. In particular, we present CasADi, an open-source software framework for numerical optimization and algorithmic differentiation (AD) that offers a level of abstraction which is lower than algebraic modeling languages, but higher than conventional AD tools. We also discuss several of the many application problems which have already been addressed with CasADi by researchers from diverse fields.

[1]  L. S. Pontryagin,et al.  Mathematical Theory of Optimal Processes , 1962 .

[2]  David D. Morrison,et al.  Multiple shooting method for two-point boundary value problems , 1962, CACM.

[3]  E B Lee,et al.  Foundations of optimal control theory , 1967 .

[4]  D. J. A. Welsh,et al.  An upper bound for the chromatic number of a graph and its application to timetabling problems , 1967, Comput. J..

[5]  L. L. Lynn,et al.  The use of orthogonal polynomials in the near-optimal control of distributed systems by trajectory approximation† , 1970 .

[6]  E. S. Parkin,et al.  Near-Optimal Control by Trajectory Approximation. Tubular Reactors with Axial Dispersion , 1970 .

[7]  M. Powell,et al.  On the Estimation of Sparse Jacobian Matrices , 1974 .

[8]  Shih-Ping Han.  A globally convergent method for nonlinear programming , 1975 .

[9]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[10]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[11]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  L. Petzold.  A description of dassl: a differential/algebraic system solver , 1982 .

[13]  P. Gill,et al.  User's Guide for SOL/NPSOL: A Fortran Package for Nonlinear Programming. , 1983 .

[14]  Lorenz T. Biegler,et al.  Simultaneous modular simulation and optimization , 1983 .

[15]  Michael A. Saunders,et al.  User''s guide for NPSOL (Ver-sion 4.0): A FORTRAN package for nonlinear programming , 1984 .

[16]  Solution of dynamic optimization problems by successive quadratic programming and orthogonal collocation , 1984 .

[17]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[18]  L. Petzold,et al.  Numerical methods and software for sensitivity analysis of differential-algebraic systems , 1986 .

[19]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[20]  C. Pantelides.  The consistent intialization of differential-algebraic systems , 1988 .

[21]  Johannes P. Schlöder,et al.  Numerische Methoden zur Behandlung hochdimensionaler Aufgaben der Parameteridentifizierung , 1988 .

[22]  J. E. Cuthrell,et al.  Simultaneous optimization and solution methods for batch reactor control profiles , 1989 .

[23]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[24]  D. Gay.  Automatic Differentiation of Nonlinear AMPL Models , 1991 .

[25]  Steven Skiena,et al.  Algorithms for Square Roots of Graphs , 1991, SIAM J. Discret. Math..

[26]  Andreas Griewank,et al.  ADIFOR - Generating Derivative Codes form Fortran Programs , 1992, Sci. Program..

[27]  E. Hairer,et al.  Solving ordinary differential equations I (2nd revised. ed.): nonstiff problems , 1993 .

[28]  Michael B. Monagan,et al.  GRADIENT: algorithmic differentiation in Maple , 1993, ISSAC '93.

[29]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[30]  Jorge Nocedal,et al.  A Reduced Hessian Method for Large-Scale Constrained Optimization , 1995, SIAM J. Optim..

[31]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[32]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[33]  H. Selbmann,et al.  Learning to recognize objects , 1999, Trends in Cognitive Sciences.

[34]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[35]  Y. Azmy.  Post-convergence automatic differentiation of iterative schemes , 1997 .

[36]  B. Christianson.  Reverse aumulation and imploicit functions , 1998 .

[37]  Thomas F. Coleman,et al.  Structured automatic differentiation , 1998 .

[38]  Michael Monagan,et al.  Automatic differentiation: an implementation in Maple , 1998 .

[39]  L. Biegler.  Advances in nonlinear programming concepts for process control , 1997 .

[40]  Stephen J. Wright,et al.  Application of Interior-Point Methods to Model Predictive Control , 1998 .

[41]  D. Mayne,et al.  Min-max feedback model predictive control for constrained linear systems , 1998, IEEE Trans. Autom. Control..

[42]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[43]  Irene Bauer.  Numerische Verfahren zur Lösung von Anfangswertaufgaben und zur Generierung von ersten und zweiten Ableitungen mit Anwendungen bei Optimierungsaufgaben in Chemie und Verfahrenstechnik , 1999 .

[44]  R. Vanderbei.  LOQO:an interior point code for quadratic programming , 1999 .

[45]  M. Diehl,et al.  Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations , 2000 .

[46]  L. Biegler,et al.  A reduced space interior point strategy for optimization of differential algebraic systems , 2000 .

[47]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[48]  Christian Bischof,et al.  Computing derivatives of computer programs , 2000 .

[49]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[50]  Andrei Alexandrescu,et al.  Modern C++ Design , 2001 .

[51]  Moritz Diehl,et al.  Real-Time Optimization for Large Scale Nonlinear Processes , 2001 .

[52]  Jay H. Lee,et al.  Constrained linear state estimation - a moving horizon approach , 2001, Autom..

[53]  Bruce A. Reed,et al.  On Star Coloring of Graphs , 2001, WG.

[54]  Andreas Griewank,et al.  Detecting Jacobian sparsity patterns by Bayesian probing , 2002, Math. Program..

[55]  O. V. Stryk.  Simulation und Systemoptimierung , 2002 .

[56]  Christian H. Bischof,et al.  Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs , 2002, Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation.

[57]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[58]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[59]  V. Torczon,et al.  A GLOBALLY CONVERGENT AUGMENTED LAGRANGIAN ALGORITHM FOR OPTIMIZATION WITH GENERAL CONSTRAINTS AND SIMPLE BOUNDS , 2002 .

[60]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[61]  Frank Allgöwer,et al.  An Introduction to Nonlinear Model Predictive Control , 2002 .

[62]  Stephen J. Wright,et al.  Object-oriented software for quadratic programming , 2003, TOMS.

[63]  Shengtai Li,et al.  Adjoint Sensitivity Analysis for Differential-Algebraic Equations: The Adjoint DAE System and Its Numerical Solution , 2002, SIAM J. Sci. Comput..

[64]  Johannes P. Schlöder,et al.  An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects , 2003, Comput. Chem. Eng..

[65]  Johannes P. Schlöder,et al.  An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization: Part II: Software aspects and applications , 2003, Comput. Chem. Eng..

[66]  David M. Beazley,et al.  Automated scientific software scripting with SWIG , 2003, Future Gener. Comput. Syst..

[67]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[68]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[69]  Peter A. Fritzson,et al.  Principles of object-oriented modeling and simulation with Modelica 2.1 , 2004 .

[70]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[71]  Lorenz T. Biegler,et al.  Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence , 2005, SIAM J. Optim..

[72]  Derya B. Özyurt,et al.  Cheap Second Order Directional Derivatives of Stiff ODE Embedded Functionals , 2005, SIAM J. Sci. Comput..

[73]  Carol S. Woodward,et al.  Enabling New Flexibility in the SUNDIALS Suite of Nonlinear and Differential/Algebraic Equation Solvers , 2020, ACM Trans. Math. Softw..

[74]  Alex Pothen,et al.  What Color Is Your Jacobian? Graph Coloring for Computing Derivatives , 2005, SIAM Rev..

[75]  David Benson,et al.  A Gauss pseudospectral transcription for optimal control , 2005 .

[76]  Lorenz T. Biegler,et al.  Line Search Filter Methods for Nonlinear Programming: Local Convergence , 2005, SIAM J. Optim..

[77]  Thomas Kaminski,et al.  Automatic Sparsity Detection Implemented as a Source-to-Source Transformation , 2006, International Conference on Computational Science.

[78]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[79]  Shaun A. Forth.  An efficient overloaded implementation of forward mode automatic differentiation in MATLAB , 2006, TOMS.

[80]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[81]  François E. Cellier,et al.  Continuous System Simulation , 2006 .

[82]  M. Diehl,et al.  Fast NMPC of a chain of masses connected by springs , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[83]  Hans Georg Bock,et al.  Sensitivity Generation in an Adaptive BDF-Method , 2006, HPSC.

[84]  Jorge Nocedal,et al.  Knitro: An Integrated Package for Nonlinear Optimization , 2006 .

[85]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[86]  Shaun A. Forth,et al.  Source Transformation for MATLAB Automatic Differentiation , 2006, International Conference on Computational Science.

[87]  Timothy A. Davis,et al.  Direct methods for sparse linear systems , 2006, Fundamentals of algorithms.

[88]  José Mario Martínez,et al.  On Augmented Lagrangian Methods with General Lower-Level Constraints , 2007, SIAM J. Optim..

[89]  James V. Burke,et al.  Algorithmic Differentiation of Implicit Functions and Optimal Values , 2008 .

[90]  Bernhard Kainz,et al.  Automatic Differentiation for GPU-Accelerated 2D/3D Registration , 2008 .

[91]  M. Giles.  An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation , 2008 .

[92]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[93]  M. Giles.  Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation , 2008 .

[94]  Manfred Morari,et al.  Real-time input-constrained MPC using fast gradient methods , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[95]  John T. Betts,et al.  Practical Methods for Optimal Control and Estimation Using Nonlinear Programming , 2009 .

[96]  Brian C. Fabien,et al.  dsoa: The implementation of a dynamic system optimization algorithm , 2010 .

[97]  Eric von Lieres,et al.  A fast and accurate solver for the general rate model of column liquid chromatography , 2010, Comput. Chem. Eng..

[98]  J. Albersmeyer.  Adjoint-based algorithms and numerical methods for sensitivity generation and optimization of large scale dynamic systems , 2010 .

[99]  Karl-Erik Årzén,et al.  Modeling and optimization with Optimica and JModelica.org - Languages and tools for solving large-scale dynamic optimization problems , 2010, Comput. Chem. Eng..

[100]  Moritz Diehl,et al.  The Lifted Newton Method and Its Application in Optimization , 2009, SIAM J. Optim..

[101]  Moritz Diehl,et al.  Towards a Computer Algebra System with Automatic Differentiation for use with Object-Oriented modelling languages , 2010, EOOLT.

[102]  Lorenz T. Biegler,et al.  Nonlinear Waves in Integrable and Nonintegrable Systems , 2018 .

[103]  Robert Pitz-Paal,et al.  Simulation and control of solar thermal power plants , 2010 .

[104]  B. Christianson.  Reverse Accumulation and Implicit Functions , 2010 .

[105]  Johannes P. Schlöder,et al.  A real-time algorithm for moving horizon state and parameter estimation , 2011, Comput. Chem. Eng..

[106]  Moritz Diehl,et al.  Multiple shooting for distributed systems with applications in hydro electricity production , 2011 .

[107]  Robert Pitz-Paal,et al.  Dynamic Simulation of a solar tower system with open volumetic receiver - a review on the vICERP project , 2011 .

[108]  Boris Houska,et al.  Robust Optimization of Dynamic Systems (Robuuste optimalisatie van dynamische systemen) , 2011 .

[109]  C. Kirches.  Fast Numerical Methods for Mixed-Integer Nonlinear Model-Predictive Control , 2011 .

[110]  Rolf Findeisen,et al.  Fast predictive control of linear systems combining Nesterov's gradient method and the method of multipliers , 2011, IEEE Conference on Decision and Control and European Control Conference.

[111]  Francesco Casella,et al.  Integration of CasADi and JModelica.org , 2011 .

[112]  Paul D. Hovland,et al.  Sparse Jacobian Computation Using ADIC2 and ColPack , 2011, ICCS.

[113]  M. Diehl,et al.  Adjoint-based distributed multiple shooting for large-scale systems , 2011 .

[114]  Moritz Diehl,et al.  An auto-generated real-time iteration algorithm for nonlinear MPC in the microsecond range , 2011, Autom..

[115]  Francesco Casella,et al.  Object-Oriented Modeling and Optimal Control: A Case Study in Power Plant Start-Up , 2011 .

[116]  David L. Woodruff,et al.  Pyomo: modeling and solving mathematical programs in Python , 2011, Math. Program. Comput..

[117]  Sebastian Engell,et al.  Multi-stage and Two-stage Robust Nonlinear Model Predictive Control , 2012 .

[118]  Fredrik Magnusson.  Collocation methods in JModelica.org , 2012 .

[119]  Moritz Diehl,et al.  Dynamic optimization with CasADi , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[120]  Piyush Grover,et al.  Optimized Three-Body Gravity Assists and Manifold Transfers in End-to-End Lunar Mission Design , 2012 .

[121]  Anders Logg,et al.  DOLFIN: a C++/Python Finite Element Library , 2012 .

[122]  Moritz Diehl,et al.  Lyapunov based design of robust linear-feedback for time-optimal periodic quadcopter motions , 2012 .

[123]  Anders Logg,et al.  Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book , 2012 .

[124]  M. Cerf,et al.  Continuation from a flat to a round Earth model in the coplanar orbit transfer problem , 2012 .

[125]  Jan Swevers,et al.  A flexible, efficient software package for nonlinear ILC , 2012 .

[126]  Moritz Diehl,et al.  Distributed Multiple Shooting for Optimal Control of Large Interconnected Systems , 2012 .

[127]  Moritz Diehl,et al.  CasADi -- A symbolic package for automatic differentiation and optimal control , 2012 .

[128]  Janick V. Frasch,et al.  Mixed–-Level Iteration Schemes for Nonlinear Model Predictive Control , 2012 .

[129]  Peter C. St. John,et al.  Identification of Small Molecule Activators of Cryptochrome , 2012, Science.

[130]  Jan Swevers,et al.  Optimal periodic control of power harvesting tethered airplanes: How to fly fast without wind and without propellor? , 2012, 2012 American Control Conference (ACC).

[131]  Quoc Tran Dinh.  Sequential Convex Programming and Decomposition Approaches for Nonlinear Optimization , 2012 .

[132]  H. Bock,et al.  Efficient direct multiple shooting for nonlinear model predictive control on long horizons , 2012 .

[133]  S. Engell,et al.  A new Robust NMPC Scheme and its Application to a Semi-batch Reactor Example* , 2012 .

[134]  Giorgio Fasano,et al.  Modeling and Optimization in Space Engineering , 2012, Springer Optimization and Its Applications.

[135]  Manfred Morari,et al.  Efficient interior point methods for multistage problems arising in receding horizon control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[136]  Sebastian Engell,et al.  Multi-stage nonlinear model predictive control applied to a semi-batch polymerization reactor under uncertainty , 2013 .

[137]  Lars Imsland,et al.  Multi-objective Predictive Control for non steady-state operation , 2013, 2013 European Control Conference (ECC).

[138]  Mario Zanon,et al.  Airborne Wind Energy Based on Dual Airfoils , 2013, IEEE Transactions on Control Systems Technology.

[139]  Moritz Diehl,et al.  A positive definiteness preserving discretization method for Lyapunov differential equations , 2013, 52nd IEEE Conference on Decision and Control.

[140]  M. Diehl,et al.  A Positive Definiteness Preserving Discretization Method for nonlinear Lyapunov Differential Equations , 2013 .

[141]  Manfred Morari,et al.  Auto-generated algorithms for nonlinear model predictive control on long and on short horizons , 2013, 52nd IEEE Conference on Decision and Control.

[142]  Goele Pipeleers,et al.  Time-optimal quadrotor flight , 2013, 2013 European Control Conference (ECC).

[143]  M. Diehl,et al.  Optimization of the Mass Flow Rate Distribution of an Open Volumetric Air Receiver , 2013 .

[144]  Goele Pipeleers,et al.  Time-optimal parking and flying: Solving path following problems efficiently , 2013, 2013 IEEE International Conference on Mechatronics (ICM).

[145]  Konstantin Nalum.  Modeling and Dynamic Optimization in Oil Production , 2013 .

[146]  Francesco Casella,et al.  A framework for nonlinear model-predictive control using object-oriented modeling with a case study in power plant start-up , 2013, 2013 IEEE Conference on Computer Aided Control System Design (CACSD).

[147]  Goele Pipeleers,et al.  Time-optimal path planning for flat systems with application to a wheeled mobile robot , 2013, 9th International Workshop on Robot Motion and Control.

[148]  Lars Imsland,et al.  Optimization-based autonomous remote sensing of surface objects using an unmanned aerial vehicle , 2013, 2013 European Control Conference (ECC).

[149]  Jan Åslund,et al.  Studying the Influence of Roll and Pitch Dynamics in Optimal Road-Vehicle Maneuvers , 2013 .

[150]  Bernhard Bachmann,et al.  Tool Demonstration Abstract: OpenModelica and CasADi for Model-Based Dynamic Optimization , 2013, EOOLT.

[151]  A CasADi Based Toolchain For JModelica.org , 2013 .

[152]  Francis J. Doyle,et al.  Estimating confidence intervals in predicted responses for oscillatory biological models , 2013, BMC Systems Biology.

[153]  D. Walraven,et al.  Optimum configuration of plate-type heat exchangers for the use in ORCs for low-temperature geothermal heat sources , 2013 .

[154]  M. Diehl,et al.  Airborne Wind Energy , 2023, Green Energy and Technology.

[155]  David A. Ham,et al.  Automated Derivation of the Adjoint of High-Level Transient Finite Element Programs , 2013, SIAM J. Sci. Comput..

[156]  Bernhard Bachmann,et al.  Model-Based Dynamic Optimization with OpenModelica and CasADi , 2013 .

[157]  Janick V. Frasch,et al.  A dual Newton strategy for the efficient solution of sparse quadratic programs arising in SQP-based nonlinear MPC , 2013 .

[158]  Sebastian Engell,et al.  Efficient Robust Economic Nonlinear Model Predictive Control of an Industrial Batch Reactor , 2014 .

[159]  W. Karush.  Minima of Functions of Several Variables with Inequalities as Side Conditions , 2014 .