Finite-time regional verification of stochastic non-linear systems

Recent trends pushing robots into unstructured environments with limited sensors have motivated considerable work on planning under uncertainty and stochastic optimal control, but these methods typically do not provide guaranteed performance. Here we consider the problem of bounding the probability of failure (defined as leaving a finite region of state space) over a finite time for stochastic non-linear systems with continuous state. Our approach searches for exponential barrier functions that provide bounds using a variant of the classical supermartingale result. We provide a relaxation of this search to a semidefinite program, yielding an efficient algorithm that provides rigorous upper bounds on the probability of failure for the original non-linear system. We give a number of numerical examples in both discrete and continuous time that demonstrate the effectiveness of the approach.

[1]  H. Kushner.  ON THE STABILITY OF STOCHASTIC DYNAMICAL SYSTEMS. , 1965, Proceedings of the National Academy of Sciences of the United States of America.

[2]  H. Kushner.  Finite time stochastic stability and the analysis of tracking systems , 1966 .

[3]  F. Beutler.  On two discrete-time system stability concepts and supermartingales , 1973 .

[4]  Tad McGeer,et al.  Passive Dynamic Walking , 1990, Int. J. Robotics Res..

[5]  Matthew R. James,et al.  Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games , 1992, Math. Control. Signals Syst..

[6]  C Yeager Jessie,et al.  Implementation and Testing of Turbulence Models for the F18-HARV Simulation , 1998 .

[7]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[8]  Peter J Seiler,et al.  SOSTOOLS: Sum of squares optimization toolbox for MATLAB , 2002 .

[9]  A. Megretski.  Positivity of trigonometric polynomials , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[10]  A. Papachristodoulou,et al.  Nonlinear control synthesis by sum of squares optimization: a Lyapunov-based approach , 2004, 2004 5th Asian Control Conference (IEEE Cat. No.04EX904).

[11]  George J. Pappas,et al.  Stochastic safety verification using barrier certificates , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[12]  A. Papachristodoulou,et al.  Analysis of Non-polynomial Systems using the Sum of Squares Decomposition , 2005 .

[13]  Pieter Abbeel,et al.  An Application of Reinforcement Learning to Aerobatic Helicopter Flight , 2006, NIPS.

[14]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[15]  Franco Blanchini,et al.  Set-theoretic methods in control , 2007 .

[16]  M. Tadjine,et al.  Sliding Mode Control Based on Backstepping Approach for an UAV Type-Quadrotor , 2007 .

[17]  James A. Primbs,et al.  Stochastic Receding Horizon Control of Constrained Linear Systems With State and Control Multiplicative Noise , 2007, IEEE Transactions on Automatic Control.

[18]  E. Westervelt,et al.  Feedback Control of Dynamic Bipedal Robot Locomotion , 2007 .

[19]  Andrew Packard,et al.  Stability Region Analysis Using Polynomial and Composite Polynomial Lyapunov Functions and Sum-of-Squares Programming , 2008, IEEE Transactions on Automatic Control.

[20]  Katie Byl,et al.  Metastable Walking on Stochastically Rough Terrain , 2008, Robotics: Science and Systems.

[21]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[22]  Stephen P. Boyd,et al.  Recent Advances in Learning and Control , 2008, Lecture Notes in Control and Information Sciences.

[23]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[24]  Russ Tedrake,et al.  Experiments in Fixed-Wing UAV Perching , 2008 .

[25]  Katie Byl,et al.  Metastable Walking Machines , 2009, Int. J. Robotics Res..

[26]  Johan Löfberg,et al.  Pre- and Post-Processing Sum-of-Squares Programs in Practice , 2009, IEEE Transactions on Automatic Control.

[27]  Ying-Xu Yang,et al.  Finite-time stability and stabilization of nonlinear stochastic hybrid systems☆ , 2009 .

[28]  Nicolas Tabareau,et al.  A Contraction Theory Approach to Stochastic Incremental Stability , 2007, IEEE Transactions on Automatic Control.

[29]  Mark M. Tobenkin,et al.  Invariant Funnels around Trajectories using Sum-of-Squares Programming , 2010, 1010.3013.

[30]  Joost-Pieter Katoen,et al.  Approximate Model Checking of Stochastic Hybrid Systems , 2010, Eur. J. Control.

[31]  Ian R. Manchester,et al.  Bounding on rough terrain with the LittleDog robot , 2011, Int. J. Robotics Res..

[32]  Jean-Pierre Aubin,et al.  Viability Theory: New Directions , 2011 .

[33]  Y. Z. Mehrjerdi.  A Chance Constrained Programming , 2012 .

[34]  M. Manhart,et al.  Markov Processes , 2018, Introduction to Stochastic Processes and Simulation.

[35]  Vijay Kumar,et al.  Trajectory generation and control for precise aggressive maneuvers with quadrotors , 2012, Int. J. Robotics Res..

[36]  E. B Dynkin,et al.  Markov Processes: Volume 1 , 2012 .

[37]  Khadir Mohamed,et al.  Model Predictive Control: Theory and Design , 2014 .