CityPM: Predictive Monitoring with Logic-Calibrated Uncertainty for Smart Cities

We present CityPM, a novel predictive monitoring system for smart cities, that continuously generates sequential predictions of future city states using Bayesian deep learning and monitors if the generated predictions satisfy city safety and performance requirements. We formally define a flowpipe signal to characterize prediction outputs of Bayesian deep learning models, and develop a new logic, named {Signal Temporal Logic with Uncertainty} (STL-U), for reasoning about the correctness of flowpipe signals. CityPM can monitor city requirements specified in STL-U such as "with 90% confidence level, the predicated air quality index in the next 10 hours should always be below 100". We also develop novel STL-U logic-based criteria to measure uncertainty for Bayesian deep learning. CityPM uses these logic-calibrated uncertainty measurements to select and tune the uncertainty estimation schema in deep learning models. We evaluate CityPM on three large-scale smart city case studies, including two real-world city datasets and one simulated city experiment. The results show that CityPM significantly improves the simulated city's safety and performance, and the use of STL-U logic-based criteria leads to improved uncertainty calibration in various Bayesian deep learning models.

[1]  Dejan Nickovic,et al.  Monitoring Temporal Properties of Continuous Signals , 2004, FORMATS/FTRTFT.

[2]  Calin Belta,et al.  SpaTeL: a novel spatial-temporal logic and its applications to networked systems , 2015, HSCC.

[3]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[4]  Abdeltawab M. Hendawi,et al.  Data Sets, Modeling, and Decision Making in Smart Cities , 2019, ACM Trans. Cyber Phys. Syst..

[5]  Alex Graves,et al.  Practical Variational Inference for Neural Networks , 2011, NIPS.

[6]  William Yang Wang,et al.  Quantifying Uncertainties in Natural Language Processing Tasks , 2018, AAAI.

[7]  Alberto L. Sangiovanni-Vincentelli,et al.  Stochastic contracts for cyber-physical system design under probabilistic requirements , 2017, MEMOCODE.

[8]  Bin Guo,et al.  CityGuard: Citywide Fire Risk Forecasting Using A Machine Learning Approach , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[9]  Dejan Nickovic,et al.  Specification-Based Monitoring of Cyber-Physical Systems: A Survey on Theory, Tools and Applications , 2018, Lectures on Runtime Verification.

[10]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[11]  Sanjit A. Seshia,et al.  Safe Autonomy Under Perception Uncertainty Using Chance-Constrained Temporal Logic , 2017, Journal of Automated Reasoning.

[12]  Alexandre Donzé,et al.  Breach, A Toolbox for Verification and Parameter Synthesis of Hybrid Systems , 2010, CAV.

[13]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[14]  Lu Feng,et al.  CityResolver: A Decision Support System for Conflict Resolution in Smart Cities , 2018, 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS).

[15]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[16]  Dejan Nickovic,et al.  Quantitative monitoring of STL with edit distance , 2016, Formal Methods in System Design.

[17]  Masamichi Shimosaka,et al.  Early Destination Prediction with Spatio-temporal User Behavior Patterns , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[18]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[19]  Yu Zheng,et al.  GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction , 2018, IJCAI.

[20]  Sriram Sankaranarayanan,et al.  S-TaLiRo: A Tool for Temporal Logic Falsification for Hybrid Systems , 2011, TACAS.

[21]  Liang Lin,et al.  Contextualized Spatial–Temporal Network for Taxi Origin-Destination Demand Prediction , 2019, IEEE Transactions on Intelligent Transportation Systems.

[22]  Kang G. Shin,et al.  Towards Fine-grained Flow Forecasting: A Graph Attention Approach for Bike Sharing Systems , 2020, WWW.

[23]  George J. Pappas,et al.  Robustness of temporal logic specifications for continuous-time signals , 2009, Theor. Comput. Sci..

[24]  Nikolay Laptev,et al.  Deep and Confident Prediction for Time Series at Uber , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[25]  Daniel Krajzewicz,et al.  SUMO - Simulation of Urban MObility An Overview , 2011 .

[26]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[27]  Fei Wu,et al.  HST-LSTM: A Hierarchical Spatial-Temporal Long-Short Term Memory Network for Location Prediction , 2018, IJCAI.

[28]  Jyotirmoy V. Deshmukh,et al.  Specification Mining and Robust Design under Uncertainty , 2019, ACM Trans. Embed. Comput. Syst..

[29]  Alex Kendall,et al.  Concrete Dropout , 2017, NIPS.

[30]  Lu Feng,et al.  SaSTL: Spatial Aggregation Signal Temporal Logic for Runtime Monitoring in Smart Cities , 2019, 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS).

[31]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[32]  Ashish Kapoor,et al.  Safe Control under Uncertainty with Probabilistic Signal Temporal Logic , 2016, Robotics: Science and Systems.

[33]  Yarin Gal,et al.  Uncertainty in Deep Learning , 2016 .