Interpretable Policies from Formally-Specified Temporal Properties

We present an approach to interpret parameterized policies through the lens of Signal Temporal Logic (STL). By providing a formally-specified description of desired behaviors we want the policy to produce, we can identify clusters in the parameter space of the policy that can produce the desired behavior. In the context of agent simulation for autonomous driving, this enables an automated way to target and produce challenging scenarios to stress-test the autonomous driving stack and hence accelerate validation and testing. Our approach leverages parametric signal temporal logic (pSTL) formulas to construct an interpretable view on the modeling parameters via a sequence of variational inference problems; one to solve for the pSTL parameters and another to construct a new parameterization satisfying the specification. We perform clustering on the new parameter space using a finite set of examples, either real or simulated, and combine computational graph learning and normalizing flows to form a relationship between these parameters and pSTL formulas either derived by hand or inferred from data. We illustrate the utility of our approach to model selection for validation of the safety properties of an autonomous driving system, using a learned generative model of the surrounding agents.

[1]  Susmit Jha,et al.  Explaining AI Decisions Using Efficient Methods for Learning Sparse Boolean Formulae , 2018, Journal of Automated Reasoning.

[2]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[3]  Stefano Ermon,et al.  InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations , 2017, NIPS.

[4]  Sanjit A. Seshia,et al.  Logic-based Clustering and Learning for Time-Series Data , 2016 .

[5]  James Kapinski,et al.  Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems , 2015, NFM.

[6]  Mykel J. Kochenderfer,et al.  Algorithms for Verifying Deep Neural Networks , 2019, Found. Trends Optim..

[7]  Marco Pavone,et al.  Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Honglak Lee,et al.  Learning Structured Output Representation using Deep Conditional Generative Models , 2015, NIPS.

[9]  Calin Belta,et al.  A Decision Tree Approach to Data Classification using Signal Temporal Logic , 2016, HSCC.

[10]  Leslie Pack Kaelbling,et al.  From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning , 2018, J. Artif. Intell. Res..

[11]  Dejan Nickovic,et al.  Parametric Identification of Temporal Properties , 2011, RV.

[12]  Calin Belta,et al.  Reinforcement learning with temporal logic rewards , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Mykel J. Kochenderfer,et al.  Factor graph scene distributions for automotive safety analysis , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[14]  Zoubin Ghahramani,et al.  Discovering Interpretable Representations for Both Deep Generative and Discriminative Models , 2018, ICML.

[15]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[16]  Mykel J. Kochenderfer,et al.  A Reachability Method for Verifying Dynamical Systems with Deep Neural Network Controllers , 2019, ArXiv.

[17]  Max Welling,et al.  Improved Variational Inference with Inverse Autoregressive Flow , 2016, NIPS 2016.

[18]  Alex Lascarides,et al.  Disentangled Relational Representations for Explaining and Learning from Demonstration , 2019, CoRL.

[19]  Houssam Abbas,et al.  Smooth operator: Control using the smooth robustness of temporal logic , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[20]  Alberto L. Sangiovanni-Vincentelli,et al.  Time Series Learning using Monotonic Logical Properties , 2018, RV.

[21]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[22]  Ashish Tiwari,et al.  TeLEx: learning signal temporal logic from positive examples using tightness metric , 2019, Formal Methods Syst. Des..

[23]  Matthias Althoff,et al.  Automatic Generation of Safety-Critical Test Scenarios for Collision Avoidance of Road Vehicles , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[24]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yang Gao,et al.  Reinforcement Learning from Imperfect Demonstrations , 2018, ICLR.

[26]  Xin Huang,et al.  DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling , 2020, IEEE Robotics and Automation Letters.

[27]  Marco Pavone,et al.  Backpropagation for Parametric STL , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).