A Distributed Optimization Algorithm for Stochastic Optimal Control

Abstract This paper presents a distributed non-convex optimization algorithm for solving stochastic optimal control problems to local optimality. Here, our focus is on a class of methods that approximates the probability distribution of the states of a stochastic optimal control problem with uncertain parameters by using a sigma point approach. This leads to a large but structured optimal control problem comprising a number of carefully selected uncertainty scenarios in order to enforce chance constraints. The approach achieves accuracies that are equivalent to a third order moment expansion. However, as the resulting large but structured optimal control problem is challenging to solve with existing numerical tools, this paper proposes a tailored distributed algorithm that exploits the particular structure that arises when applying the sigma point approach. The method is based on a tailored variant of the recently proposed augmented Lagrangian based alternating direction inexact Newton (ALADIN) algorithm. The approach is illustrated by the application to a benchmark case study involving a predator-prey-fishing model.

[1]  Sandro Macchietto,et al.  Designing robust optimal dynamic experiments , 2002 .

[2]  Lars Grüne,et al.  Numerical Methods for Optimal Control Problems , 2018 .

[3]  Filip Logist,et al.  Approximate robust optimization of nonlinear systems under parametric uncertainty and process noise , 2015 .

[4]  N. Cutland,et al.  On homogeneous chaos , 1991, Mathematical Proceedings of the Cambridge Philosophical Society.

[5]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[6]  Jan Van Impe,et al.  Robust Optimization of Nonlinear Dynamic Systems with Application to a Jacketed Tubular Reactor , 2012 .

[7]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[8]  Nikolaos V. Sahinidis,et al.  Optimization under uncertainty: state-of-the-art and opportunities , 2004, Comput. Chem. Eng..

[9]  Filip Logist,et al.  Multi-objective optimal control of dynamic bioprocesses using ACADO Toolkit , 2013, Bioprocess and Biosystems Engineering.

[10]  Johan A. K. Suykens,et al.  Distributed nonlinear optimal control using sequential convex programming and smoothing techniques , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[11]  U. Diwekar,et al.  Efficient sampling technique for optimization under uncertainty , 1997 .

[12]  Richard D. Braatz,et al.  Stochastic nonlinear model predictive control with probabilistic constraints , 2014, 2014 American Control Conference.

[13]  S. Julier,et al.  A General Method for Approximating Nonlinear Transformations of Probability Distributions , 1996 .

[14]  Richard D. Braatz,et al.  Open-loop and closed-loop robust optimal control of batch processes using distributional and worst-case analysis , 2004 .

[15]  Johannes P. Schlöder,et al.  Numerical methods for optimal control problems in design of robust optimal experiments for nonlinear dynamic processes , 2004, Optim. Methods Softw..

[16]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[17]  Dominique Bonvin,et al.  Dynamic optimization of batch processes: II. Role of measurements in handling uncertainty , 2003, Comput. Chem. Eng..

[18]  Moritz Diehl,et al.  An Augmented Lagrangian Based Algorithm for Distributed NonConvex Optimization , 2016, SIAM J. Optim..

[19]  S. Engell,et al.  A new Robust NMPC Scheme and its Application to a Semi-batch Reactor Example* , 2012 .

[20]  Sebastian Sager,et al.  Sampling Decisions in Optimum Experimental Design in the Light of Pontryagin's Maximum Principle , 2013, SIAM J. Control. Optim..

[21]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[22]  Tong Zhang,et al.  Medium-term maintenance turnaround planning under uncertainty for integrated chemical sites , 2016, Comput. Chem. Eng..

[23]  M. Wendt,et al.  Nonlinear Chance-Constrained Process Optimization under Uncertainty , 2002 .

[24]  Jan Van Impe,et al.  Robust multi-objective optimal control of uncertain (bio)chemical processes , 2011 .

[25]  Massimiliano Barolo,et al.  A backoff strategy for model‐based experiment design under parametric uncertainty , 2009 .

[26]  Harvey J. Everett.  Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources , 1963 .