SOTER: A Runtime Assurance Framework for Programming Safe Robotics Systems

The recent drive towards achieving greater autonomy and intelligence in robotics has led to high levels of complexity. Autonomous robots increasingly depend on third-party off-the-shelf components and complex machine-learning techniques. This trend makes it challenging to provide strong design-time certification of correct operation. To address these challenges, we present SOTER, a robotics programming framework with two key components: (1) a programming language for implementing and testing high-level reactive robotics software, and (2) an integrated runtime assurance (RTA) system that helps enable the use of uncertified components, while still providing safety guarantees. SOTER provides language primitives to declaratively construct a RTA module consisting of an advanced, high-performance controller (uncertified), a safe, lower-performance controller (certified), and the desired safety specification. The framework provides a formal guarantee that a well-formed RTA module always satisfies the safety specification, without completely sacrificing performance by using higher performance uncertified components whenever safe. SOTER allows the complex robotics software stack to be constructed as a composition of RTA modules, where each uncertified component is protected using a RTA module. To demonstrate the efficacy of our framework, we consider a real-world case-study of building a safe drone surveillance system. Our experiments both in simulation and on actual drones show that the SOTER-enabled RTA ensures the safety of the system, including when untrusted third-party components have bugs or deviate from the desired behavior.

[1]  Nancy A. Lynch,et al.  An introduction to input/output automata , 1989 .

[2]  Sanjit A. Seshia,et al.  Combining Model Checking and Runtime Verification for Safe Robotics , 2017, RV.

[3]  John D. Schierman,et al.  A Component-Based Simplex Architecture for High-Assurance Cyber-Physical Systems , 2017, 2017 17th International Conference on Application of Concurrency to System Design (ACSD).

[4]  John D. Schierman,et al.  Runtime Assurance Framework Development for Highly Adaptive Flight Control Systems , 2015 .

[5]  Paulo Tabuada,et al.  Linear temporal logic motion planning for teams of underactuated robots using satisfiability modulo convex programming , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[6]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[7]  Insup Lee,et al.  A Study on Run Time Assurance for Complex Cyber Physical Systems , 2013 .

[8]  Bruno Dutertre,et al.  Modeling and Verification of a Fault-Tolerant Real-Time Startup Protocol Using Calendar Automata , 2004, FORMATS/FTRTFT.

[9]  Martin Törngren,et al.  Tuning Permissiveness of Active Safety Monitors for Autonomous Systems , 2018, NFM.

[10]  Brian Charles Williams,et al.  Robust Execution of Temporally Flexible Plans for Bipedal Walking Devices , 2006, ICAPS.

[11]  Jérémie Guiochet,et al.  Safety-critical advanced robots: A survey , 2017, Robotics Auton. Syst..

[12]  Sanjit A. Seshia,et al.  Towards Verified Artificial Intelligence , 2016, ArXiv.

[13]  Lavindra de Silva,et al.  A Verifiable and Correct-by-Construction Controller for Robot Functional Levels , 2011, ArXiv.

[14]  Sanjit A. Seshia,et al.  DRONA: A Framework for Safe Distributed Mobile Robotics , 2017, 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS).

[15]  Yi Zhang,et al.  ROSRV: Runtime Verification for Robots , 2014, RV.

[16]  Lavindra de Silva,et al.  Rigorous design of robot software: A formal component-based approach , 2012, Robotics Auton. Syst..

[17]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, NFM.

[18]  Ola Pettersson,et al.  Execution monitoring in robotics: A survey , 2005, Robotics Auton. Syst..

[19]  Lui Sha,et al.  Using Simplicity to Control Complexity , 2001, IEEE Softw..

[20]  Radu Grosu,et al.  Collision avoidance for mobile robots with limited sensing and limited information about moving obstacles , 2017, Formal Methods Syst. Des..

[21]  Brian C. Williams,et al.  Generative Planning for Hybrid Systems Based on Flow Tubes , 2008, ICAPS.

[22]  Sanjit A. Seshia,et al.  Programming Safe Robotics Systems: Challenges and Advances , 2018, ISoLA.

[23]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[24]  Claire J. Tomlin,et al.  Extensions of learning-based model predictive control for real-time application to a quadrotor helicopter , 2012, 2012 American Control Conference (ACC).

[25]  Mahesh Viswanathan,et al.  C2E2: A Verification Tool for Stateflow Models , 2015, TACAS.

[26]  Garvit Juniwal,et al.  Robust online monitoring of signal temporal logic , 2015, Formal Methods in System Design.

[27]  Thomas A. Henzinger,et al.  Reactive Modules , 1996, Proceedings 11th Annual IEEE Symposium on Logic in Computer Science.

[28]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[29]  Vijay Kumar,et al.  Automated composition of motion primitives for multi-robot systems from safe LTL specifications , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[31]  Mo Chen,et al.  FaSTrack: A modular framework for fast and guaranteed safe motion planning , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[32]  Danbing Seto,et al.  Case Study: Development of a Baseline Controller for Automatic Landing of an F-16 Aircraft Using Linear Matrix Inequalities (LMIs) , 2000 .

[33]  Jaime F. Fisac,et al.  Reachability-based safe learning with Gaussian processes , 2014, 53rd IEEE Conference on Decision and Control.

[34]  Lui Sha,et al.  The System-Level Simplex Architecture for Improved Real-Time Embedded System Safety , 2009, 2009 15th IEEE Real-Time and Embedded Technology and Applications Symposium.

[35]  Ashish Tiwari,et al.  SOTER: Programming Safe Robotics System using Runtime Assurance , 2018, ArXiv.

[36]  Damien Zufferey,et al.  P: safe asynchronous event-driven programming , 2013, PLDI.

[37]  S. Shankar Sastry,et al.  Provably safe and robust learning-based model predictive control , 2011, Autom..

[38]  Marco Caccamo,et al.  Sandboxing Controllers for Cyber-Physical Systems , 2011, 2011 IEEE/ACM Second International Conference on Cyber-Physical Systems.

[39]  Hadas Kress-Gazit,et al.  Temporal-Logic-Based Reactive Mission and Motion Planning , 2009, IEEE Transactions on Robotics.

[40]  André Platzer,et al.  ModelPlex: verified runtime validation of verified cyber-physical system models , 2014, Formal Methods in System Design.

[41]  Sanjit A. Seshia,et al.  Compositional programming and testing of dynamic distributed systems , 2018, Proc. ACM Program. Lang..

[42]  Mo Chen,et al.  Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems , 2016, IEEE Transactions on Automatic Control.

[43]  Alexandre M. Bayen,et al.  A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games , 2005, IEEE Transactions on Automatic Control.

[44]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[45]  Lydia E. Kavraki,et al.  The Open Motion Planning Library , 2012, IEEE Robotics & Automation Magazine.

[46]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[47]  Pieter Abbeel,et al.  Safe Exploration in Markov Decision Processes , 2012, ICML.

[48]  Ufuk Topcu,et al.  TuLiP: a software toolbox for receding horizon temporal logic planning , 2011, HSCC '11.

[49]  André Platzer,et al.  VeriPhy: verified controller executables from verified cyber-physical system models , 2018, PLDI.

[50]  Hanêne Ben-Abdallah,et al.  A Monitoring and Checking Framework for Run-time Correctness Assurance , 1998 .

[51]  Hadas Kress-Gazit,et al.  LTLMoP: Experimenting with language, Temporal Logic and robot control , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[52]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[53]  Thomas A. Henzinger,et al.  Bounded Asynchrony: Concurrency for Modeling Cell-Cell Interactions , 2008, FMSB.