Backpropagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods

This paper presents a technique, named STLCG, to compute the quantitative semantics of Signal Temporal Logic (STL) formulas using computation graphs. STLCG provides a platform which enables the incorporation of logical specifications into robotics problems that benefit from gradient-based solutions. Specifically, STL is a powerful and expressive formal language that can specify spatial and temporal properties of signals generated by both continuous and hybrid systems. The quantitative semantics of STL provide a robustness metric, i.e., how much a signal satisfies or violates an STL specification. In this work, we devise a systematic methodology for translating STL robustness formulas into computation graphs. With this representation, and by leveraging off-the-shelf automatic differentiation tools, we are able to back-propagate through STL robustness formulas and hence enable a natural and easy-to-use integration with many gradient-based approaches used in robotics. We demonstrate, through examples stemming from various robotics applications, that STLCG is versatile, computationally efficient, and capable of injecting human-domain knowledge into the problem formulation.

[1]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[2]  Houssam Abbas,et al.  Smooth operator: Control using the smooth robustness of temporal logic , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[3]  Marco Pavone,et al.  Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach , 2021, IEEE Robotics and Automation Letters.

[4]  Marco Pavone,et al.  Multi-objective optimal control for proactive decision making with temporal logic models , 2019, Int. J. Robotics Res..

[5]  Marco Pavone,et al.  Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Georgios Fainekos,et al.  Worst-case Satisfaction of STL Specifications Using Feedforward Neural Network Controllers , 2019, ACM Trans. Embed. Comput. Syst..

[7]  Calin Belta,et al.  Arithmetic-Geometric Mean Robustness for Control from Signal Temporal Logic Specifications , 2019, 2019 American Control Conference (ACC).

[8]  S. Shankar Sastry,et al.  Stochastic predictive freeway ramp metering from Signal Temporal Logic specifications , 2017, 2017 American Control Conference (ACC).

[9]  Mykel J. Kochenderfer,et al.  Algorithms for Verifying Deep Neural Networks , 2019, Found. Trends Optim..

[10]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[11]  Jyotirmoy V. Deshmukh,et al.  Learning from Demonstrations using Signal Temporal Logic , 2021, CoRL.

[12]  Hadas Kress-Gazit,et al.  LTLMoP: Experimenting with language, Temporal Logic and robot control , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  John A. Stankovic,et al.  STLnet: Signal Temporal Logic Enforced Multivariate Recurrent Neural Networks , 2020, NeurIPS.

[14]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[15]  Sriram Sankaranarayanan,et al.  Verification of automotive control applications using S-TaLiRo , 2012, 2012 American Control Conference (ACC).

[16]  Sanjit A. Seshia,et al.  Logical Clustering and Learning for Time-Series Data , 2016, CAV.

[17]  Ashish Kapoor,et al.  Safe Control under Uncertainty with Probabilistic Signal Temporal Logic , 2016, Robotics: Science and Systems.

[18]  Dejan Nickovic,et al.  Specification-Based Monitoring of Cyber-Physical Systems: A Survey on Theory, Tools and Applications , 2018, Lectures on Runtime Verification.

[19]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[20]  Christel Baier,et al.  Principles of model checking , 2008 .

[21]  Hadas Kress-Gazit,et al.  Synthesis for Robots: Guarantees and Feedback for Robot Behavior , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[22]  Calin Belta,et al.  A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks , 2018, 2018 Annual American Control Conference (ACC).

[23]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[24]  Calin Belta,et al.  Recurrent Neural Network Controllers for Signal Temporal Logic Specifications Subject to Safety Constraints , 2020, IEEE Control Systems Letters.

[25]  Calin Belta,et al.  Reinforcement learning with temporal logic rewards , 2016, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Calin Belta,et al.  Q-Learning for robust satisfaction of signal temporal logic specifications , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[27]  Dejan Nickovic,et al.  Parametric Identification of Temporal Properties , 2011, RV.

[28]  Zoubin Ghahramani,et al.  Discovering Interpretable Representations for Both Deep Generative and Discriminative Models , 2018, ICML.

[29]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[30]  George J. Pappas,et al.  Temporal logic motion planning for dynamic robots , 2009, Autom..

[31]  GhiasiSoheil,et al.  Optode Design Space Exploration for Clinically-robust Non-invasive Fetal Oximetry , 2019 .

[32]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[33]  Alberto L. Sangiovanni-Vincentelli,et al.  Model predictive control with signal temporal logic specifications , 2014, 53rd IEEE Conference on Decision and Control.

[34]  Ufuk Topcu,et al.  TuLiP: a software toolbox for receding horizon temporal logic planning , 2011, HSCC '11.