Neural Network for Weighted Signal Temporal Logic

In this paper, we propose a neuro-symbolic framework called weighted Signal Temporal Logic Neural Network (wSTL-NN) that combines the characteristics of neural networks and temporal logics. Weighted Signal Temporal Logic (wSTL) formulas are recursively composed of subformulas that are combined using logical and temporal operators. The quantitative semantics of wSTL is defined such that the quantitative satisfaction of subformulas with higher weights has more influence on the quantitative satisfaction of the overall wSTL formula. In the wSTL-NN, each neuron corresponds to a wSTL subformula, and its output corresponds to the quantitative satisfaction of the formula. We use wSTL-NN to represent wSTL formulas as features to classify time series data. STL features are more explainable than those used in classical methods. The wSTL-NN is end-to-end differentiable, which allows learning of wSTL formulas to be done using back-propagation. To reduce the number of weights, we introduce two techniques to sparsify the wSTL-NN.We apply our framework to an occupancy detection time-series dataset to learn a classifier that predicts the occupancy status of an office room.

[1]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[2]  Calin Belta,et al.  Temporal logic inference for classification and prediction from data , 2014, HSCC.

[3]  Nilanjan Ray,et al.  Pattern Recognition Letters , 1995 .

[4]  IEEE Access , 2021, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[5]  Michael M. Zavlanos,et al.  STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot Systems , 2018, Int. J. Robotics Res..

[6]  Ufuk Topcu,et al.  Verifiable RNN-Based Policies for POMDPs Under Temporal Logic Constraints , 2020, IJCAI.

[7]  Zhe Xu,et al.  Swarm Signal Temporal Logic Inference for Swarm Behavior Analysis , 2019, IEEE Robotics and Automation Letters.

[8]  Alexander G. Gray,et al.  Logical Neural Networks , 2020, ArXiv.

[9]  De Leone,et al.  Computational Optimization and Applications Volume 34, Number 2, June 2006 , 2006 .

[10]  Dimos V. Dimarogonas,et al.  Barrier Function Based Collaborative Control of Multiple Robots Under Signal Temporal Logic Tasks , 2020, IEEE Transactions on Control of Network Systems.

[11]  Ufuk Topcu,et al.  Transfer of Temporal Logic Formulas in Reinforcement Learning , 2019, IJCAI.

[12]  Fan Yang,et al.  Differentiable Learning of Logical Rules for Knowledge Base Reasoning , 2017, NIPS.

[13]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[14]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[15]  Artur S. d'Avila Garcez,et al.  Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge , 2016, NeSy@HLAI.

[16]  Naomi Ehrich Leonard,et al.  IEEE Transactions on Control of Network Systems , 2018, IEEE Transactions on Control of Network Systems.

[17]  Calin Belta,et al.  Specifying User Preferences using Weighted Signal Temporal Logic , 2020, 2021 American Control Conference (ACC).

[18]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[19]  Hai Lin,et al.  Automatic Trajectory Synthesis for Real-Time Temporal Logic , 2020, ArXiv.

[20]  Germain Forestier,et al.  Deep learning for time series classification: a review , 2018, Data Mining and Knowledge Discovery.

[21]  Kien Ming Ng,et al.  An algorithm for nonlinear optimization problems with binary variables , 2010, Comput. Optim. Appl..

[22]  Oded Maler,et al.  Robust Satisfaction of Temporal Logic over Real-Valued Signals , 2010, FORMATS.

[23]  Hadas Kress-Gazit,et al.  Event-Based Signal Temporal Logic Synthesis for Single and Multi-Robot Tasks , 2020, IEEE Robotics and Automation Letters.

[24]  Luis M. Candanedo,et al.  Accurate occupancy detection of an office room from light, temperature, humidity and CO2 measurements using statistical learning models , 2016 .

[25]  R. Venkatesh Babu,et al.  Training Sparse Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).