Accelerated Learning of Predictive Runtime Monitors for Rare Failure

Predictive runtime verification estimates the probability of a future event by monitoring the executions of a system. In this paper we use Discrete-Time Markov Chains (DTMC) as predictive models that are trained from many execution samples demonstrating a rare event: an event that occurs with very low probability. More specifically, we propose a method of grammar inference by which a DTMC is learned with far fewer samples than normal sample distribution. We exploit the concept of importance sampling, and use a mixture of samples, generated from the original system distribution and distributions that are suitably modified to produce more failures. Using the likelihood ratios of the various samples, we ensure the final trained model is faithful to the original distribution. In this way we construct accurate predictive models with orders of magnitude fewer samples. We demonstrate the gains of our approach on a file transmission protocol case study from the literature, and highlight future directions.

[1]  Christian R. Shelton,et al.  Importance sampling for reinforcement learning with multiple objectives , 2001 .

[2]  Christel Baier,et al.  Rare-event verification for stochastic hybrid systems , 2012, HSCC '12.

[3]  Marius Bozga,et al.  Improved Learning for Stochastic Timed Models by State-Merging Algorithms , 2017, NFM.

[4]  Leslie Pack Kaelbling,et al.  Learning Policies with External Memory , 1999, ICML.

[5]  A. Kolmogoroff Confidence Limits for an Unknown Distribution Function , 1941 .

[6]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.

[7]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[8]  Sanjoy Dasgupta,et al.  Off-Policy Temporal Difference Learning with Function Approximation , 2001, ICML.

[9]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[10]  Henrik Ejersbo Jensen,et al.  Reachability Analysis of Probabilistic Systems by Successive Refinements , 2001, PAPM-PROBMIV.

[11]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[12]  Martin Leucker,et al.  The Good, the Bad, and the Ugly, But How Ugly Is Ugly? , 2007, RV.

[13]  Arie Gurfinkel,et al.  Prevent : A Predictive Run-Time Verification Framework Using Statistical Learning , 2018, SEFM.

[14]  Colin de la Higuera,et al.  Grammatical Inference: Learning Automata and Grammars , 2010 .

[15]  Cyrille Jégourel,et al.  A Platform for High Performance Statistical Model Checking - PLASMA , 2012, TACAS.

[16]  Serge Haddad,et al.  Coupling and Importance Sampling for Statistical Model Checking , 2012, TACAS.

[17]  Cyrille Jégourel,et al.  Cross-Entropy Optimisation of Importance Sampling Parameters for Statistical Model Checking , 2012, CAV.

[18]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo Method (Wiley Series in Probability and Statistics) , 1981 .

[19]  A. Prasad Sistla,et al.  Monitorability of Stochastic Dynamical Systems , 2011, CAV.

[20]  Haym Hirsh,et al.  Learning to Predict Rare Events in Event Sequences , 1998, KDD.

[21]  Cyrille Jégourel,et al.  Command-based importance sampling for statistical model checking , 2016, Theor. Comput. Sci..

[22]  Frits W. Vaandrager,et al.  Proof-Checking a Data Link Protocol , 1994, TYPES.

[23]  Kim G. Larsen,et al.  Learning Probabilistic Automata for Model Checking , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[24]  Martin Leucker,et al.  Comparing LTL Semantics for Runtime Verification , 2010, J. Log. Comput..

[25]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[26]  Cyrille Jégourel,et al.  Importance Splitting for Statistical Model Checking Rare Properties , 2013, CAV.

[27]  Oded Maler,et al.  Some Thoughts on Runtime Verification , 2016, RV.

[28]  Martin Leucker,et al.  A brief account of runtime verification , 2009, J. Log. Algebraic Methods Program..

[29]  Cyrille Jégourel,et al.  An Effective Heuristic for Adaptive Importance Splitting in Statistical Model Checking , 2014, ISoLA.

[30]  Yliès Falcone,et al.  Runtime Verification of Safety-Progress Properties , 2009, RV.

[31]  Gerardo Rubino,et al.  Rare Event Simulation using Monte Carlo Methods , 2009 .

[32]  José Oncina,et al.  Learning Stochastic Regular Grammars by Means of a State Merging Method , 1994, ICGI.

[33]  Arie Gurfinkel,et al.  Predictive Run-Time Verification of Discrete-Time Reachability Properties in Black-Box Systems Using Trace-Level Abstraction and Statistical Learning , 2018, RV.

[34]  Axel Legay,et al.  PLASMA-lab: A Flexible, Distributable Statistical Model Checking Library , 2013, QEST.

[35]  Leonid Peshkin,et al.  Learning from Scarce Experience , 2002, ICML.