Control-Tree Optimization: an approach to MPC under discrete Partial Observability

This paper presents a new approach to Model Predictive Control for environments where essential, discrete variables are partially observed. Under this assumption, the belief state is a probability distribution over a finite number of states. We optimize a control-tree where each branch assumes a given state-hypothesis. The control-tree optimization uses the probabilistic belief state information. This leads to policies more optimized with respect to likely states than unlikely ones, while still guaranteeing robust constraint satisfaction at all times. We apply the method to both linear and non-linear MPC with constraints. The optimization of the control-tree is decomposed into optimization subproblems that are solved in parallel leading to good scalability for high number of state-hypotheses. We demonstrate the real-time feasibility of the algorithm on two examples and show the benefits compared to a classical MPC scheme optimizing w.r.t. one single hypothesis.

[1]  Sebastian Engell,et al.  Multi-stage nonlinear model predictive control applied to a semi-batch polymerization reactor under uncertainty , 2013 .

[2]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[3]  Moritz Diehl,et al.  Handling uncertainty in economic nonlinear model predictive control: A comparative case study , 2014 .

[4]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).

[5]  Joel A. Paulson,et al.  Stochastic model predictive control - how does it work? , 2017, Comput. Chem. Eng..

[6]  Moritz Diehl,et al.  A dual Newton strategy for scenario decomposition in robust multistage MPC , 2018 .

[7]  Andreas Potschka,et al.  Dual decomposition for QPs in scenario tree NMPC , 2015, 2015 European Control Conference (ECC).

[8]  S. Engell,et al.  A new Robust NMPC Scheme and its Application to a Semi-batch Reactor Example* , 2012 .

[9]  Marc Toussaint,et al.  Combined Task and Motion Planning under Partial Observability: An Optimization-Based Approach , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[10]  Marc Toussaint.  A Novel Augmented Lagrangian Approach for Inequalities and Convergent Any-Time Non-Central Updates , 2014, 1412.4329.

[11]  Marc Toussaint.  A Tutorial on Newton Methods for Constrained Trajectory Optimization and Relations to SLAM, Gaussian Process Smoothing, Optimal Control, and Probabilistic Inference , 2017, Geometric and Numerical Foundations of Movements.

[12]  A. Mesbah,et al.  Stochastic Model Predictive Control: An Overview and Perspectives for Future Research , 2016, IEEE Control Systems.

[13]  Sebastien Gros,et al.  A Parallelizable Interior Point Method for Two-Stage Robust MPC , 2017, IEEE Transactions on Control Systems Technology.

[14]  Rainer Manuel Schaich,et al.  Robust model predictive control , 2017 .

[15]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[16]  Moritz Diehl,et al.  Optimization Algorithms for Model Predictive Control , 2015, Encyclopedia of Systems and Control.

[17]  Marc Toussaint,et al.  Newton methods for k-order Markov Constrained Motion Problems , 2014, ArXiv.

[18]  P. Toint,et al.  A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds , 1991 .

[19]  Sebastian Engell,et al.  Robust NMPC Schemes for the Control of Mobile Robots in the Presence of Dynamic Obstacles , 2018, 2018 23rd International Conference on Methods & Models in Automation & Robotics (MMAR).