Optimization-based approximations to stochastic reachability problems

In this thesis we focus on a particular class of optimal control problems where the objective is to steer a dynamical system affected by endogenous or exogenous stochastic uncertainties to a set of target states while avoiding states which are unsafe. Given estimates of how uncertainty affects the system evolution, one can quantify the probability that trajectories satisfy such performance and safety specifications using a stochastic reachability framework. Potential applications of this framework include robotic systems, financial markets, climate models, and other problems where the state space is typically split into safe and unsafe regions. We restrict ourselves to a subclass of stochastic reachability problems, namely the finite horizon reach-avoid problem for discrete-time continuous space Markov chains. The solution to reachability problems for this class of systems is known to be given by a dynamic programming recursion that is commonly solved using space gridding. Gridding methods provide explicit error bounds for the constructed solutions but the grid size grows exponentially as a function of the space dimension when one is interested in keeping the error small. We present novel methods based on tools from randomized and robust convex optimization to approximate infinite dimensional linear programs that are shown to be equivalent to reach-avoid recursions. Our methods provide weaker bounds on the approximation error compared to space gridding but scale to problems of higher dimension. We provide several numerical examples to support this claim which is a consequence of the fact that our approximation methods rely on solving convex programs whose complexity is known to grow polynomially in the dimension of the decision and constraint space. In order to demonstrate both the generality of reach-avoid problems, as well as the limitations of solving them via space gridding, we use a case study in autonomous surveillance and develop a target tracking and acquisition framework based solely on reach-avoid recursions. We illustrate on an experimental platform with industrial surveillance cameras that the framework is successfully carrying out surveillance tasks but is limited to simple system models due to the curse of dimensionality that affects space gridding algorithms. To address these limitations, we employ the approximation algorithms developed throughout the thesis and extend the scope of the considered surveillance tasks.