Safe AI for CPS (Invited Paper)

Autonomous cyber-physical systems-such as self-driving cars and autonomous drones-often leverage artificial intelligence and machine learning algorithms to act well in open environments. Although testing plays an important role in ensuring safety and robustness, modern autonomous systems have grown so complex that achieving safety via testing alone is intractable. Formal verification reduces this testing burden by ruling out large classes of errant behavior at design time. This paper reviews recent work toward developing formal methods for cyber-physical systems that use AI for planning and control by combining the rigor of formal proofs with the flexibility of reinforcement learning.

[1]  André Platzer,et al.  A Complete Uniform Substitution Calculus for Differential Dynamic Logic , 2016, Journal of Automated Reasoning.

[2]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[3]  Junfeng Yang,et al.  Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems , 2017, ArXiv.

[4]  André Platzer,et al.  VeriPhy: verified controller executables from verified cyber-physical system models , 2018, PLDI.

[5]  Nathan Fulton,et al.  Bellerophon: Tactical Theorem Proving for Hybrid Systems , 2017, ITP.

[6]  Johannes Hölzl,et al.  Markov Chains and Markov Decision Processes in Isabelle/HOL , 2017, Journal of Automated Reasoning.

[7]  Martin Fränzle,et al.  SAT Modulo ODE: A Direct SAT Approach to Hybrid Systems , 2008, ATVA.

[8]  Nathan Fulton,et al.  Safe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning , 2018, AAAI.

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Marcelo H. Ang,et al.  Perception, Planning, Control, and Coordination for Autonomous Vehicles , 2017 .

[11]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[12]  Thomas A. Henzinger,et al.  Hybrid Automata: An Algorithmic Approach to the Specification and Verification of Hybrid Systems , 1992, Hybrid Systems.

[13]  Anil Nerode,et al.  Models for Hybrid Systems: Automata, Topologies, Controllability, Observability , 1992, Hybrid Systems.

[14]  Claire J. Tomlin,et al.  Guaranteed Safe Online Learning via Reachability: tracking a ground target using a quadrotor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[15]  Ufuk Topcu,et al.  Safe Reinforcement Learning via Shielding , 2017, AAAI.

[16]  Pushmeet Kohli,et al.  Training verified learners with learned verifiers , 2018, ArXiv.

[17]  André Platzer,et al.  ModelPlex: verified runtime validation of verified cyber-physical system models , 2014, Formal Methods in System Design.

[18]  Daniel Kroening,et al.  Logically-Correct Reinforcement Learning , 2018, ArXiv.

[19]  André Platzer,et al.  Differential Dynamic Logic for Hybrid Systems , 2008, Journal of Automated Reasoning.

[20]  André Platzer,et al.  Logics of Dynamical Systems , 2012, 2012 27th Annual IEEE Symposium on Logic in Computer Science.

[21]  Wei Chen,et al.  dReach: δ-Reachability Analysis for Hybrid Systems , 2015, TACAS.

[22]  Jaime F. Fisac,et al.  A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems , 2017, IEEE Transactions on Automatic Control.

[23]  Nathan Fulton,et al.  KeYmaera X: An Axiomatic Tactical Theorem Prover for Hybrid Systems , 2015, CADE.

[24]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.