Runtime Verification: Passing on the Baton

Twenty years have passed since the first workshop on runtime verification — the area has grown and evolved with hundreds of papers published and a sizeable number of mature tools developed. In a special occasion like this it is good to look back, but it is also good to look forward to the future. In this paper, we outline a very brief history of runtime verification, and propose a way of passing the knowledge down to future generations of academics and industry practitioners in the form of a roadmap for teaching runtime verification. The proposal, based on our experience, not only equips students with the fundamental theory underpinning runtime verification, but also ensures they have the required skills to engineer it into industrial systems. Our hope is that this would increase uptake and eventually give rise to the establishment of industry-grade tools.

[1]  Runtime Verification , 2010, Lecture Notes in Computer Science.

[2]  Klaus Havelund,et al.  Rule-based runtime verification revisited , 2015, International Journal on Software Tools for Technology Transfer.

[3]  Amir Pnueli,et al.  PSL Model Checking and Run-Time Verification Via Testers , 2006, FM.

[4]  Gordon J. Pace,et al.  Dynamic Event-Based Runtime Monitoring of Real-Time and Contextual Properties , 2009, FMICS.

[5]  César Sánchez,et al.  Gray-box Monitoring of Hyperproperties , 2019 .

[6]  Martin Leucker,et al.  Runtime Verification for LTL and TLTL , 2011, TSEM.

[7]  Ezio Bartocci,et al.  First international Competition on Runtime Verification: rules, benchmarks, tools, and final results of CRV 2014 , 2017, International Journal on Software Tools for Technology Transfer.

[8]  Bernd Finkbeiner,et al.  Temporal Logics for Hyperproperties , 2013, POST.

[9]  Grigore Rosu,et al.  Synthesizing Monitors for Safety Properties , 2002, TACAS.

[10]  Doron Drusinsky,et al.  The Temporal Rover and the ATG Rover , 2000, SPIN.

[11]  Wolfgang Ahrendt,et al.  Testing Meets Static and Runtime Verification , 2018, 2018 IEEE/ACM 6th International FME Workshop on Formal Methods in Software Engineering (FormaliSE).

[12]  Martin Leucker,et al.  jUnitRV-Adding Runtime Verification to jUnit , 2013, NASA Formal Methods.

[13]  Jerry J. Harrow.  Runtime Checking of Multithreaded Applications with Visual Threads , 2000, SPIN.

[14]  Howard Barringer,et al.  Quantified Event Automata: Towards Expressive and Efficient Runtime Monitors , 2012, FM.

[15]  César Sánchez,et al.  Gray-box Monitoring of Hyperproperties (Extended Version) , 2019, FM.

[16]  Bernd Finkbeiner,et al.  Monitoring Hyperproperties , 2017, RV.

[17]  Insup Lee,et al.  Simulation of Simultaneous Events in Regular Expressions for Run-Time Verification , 2004, RV@ETAPS.

[18]  Janusz A. Brzozowski,et al.  Derivatives of Regular Expressions , 1964, JACM.

[19]  Dimitra Giannakopoulou,et al.  Automata-based verification of temporal properties on running programs , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[20]  Eric Bodden,et al.  Clara: Partially Evaluating Runtime Monitors at Compile Time - Tutorial Supplement , 2010, RV.

[21]  Gordon J. Pace,et al.  Control-Flow Residual Analysis for Symbolic Automata , 2017, PrePost@iFM.

[22]  Gordon J. Pace,et al.  Industrial Experiences with Runtime Verification of Financial Transaction Systems: Lessons Learnt and Standing Challenges , 2018, Lectures on Runtime Verification.

[23]  Umair Siddique,et al.  Rewriting-Based Runtime Verification for Alternation-Free HyperLTL , 2017, TACAS.

[24]  Gordon J. Pace,et al.  Considering Academia-Industry Projects Meta-characteristics in Runtime Verification Design , 2018, ISoLA.

[25]  Gordon J. Pace,et al.  StaRVOOrS: A Tool for Combined Static and Runtime Verification of Java , 2015, RV.

[26]  Gordon J. Pace,et al.  Verifying data- and control-oriented properties combining static and runtime verification: theory and tools , 2017, Formal Methods Syst. Des..

[27]  Gordon J. Pace,et al.  Safer asynchronous runtime monitoring using compensations , 2012, Formal Methods Syst. Des..

[28]  Bernd Finkbeiner,et al.  The Complexity of Monitoring Hyperproperties , 2018, 2018 IEEE 31st Computer Security Foundations Symposium (CSF).

[29]  Yliès Falcone,et al.  Runtime Verification of Safety-Progress Properties , 2009, RV.

[30]  Grigore Rosu,et al.  Runtime Verification - 17 Years Later , 2018, RV.

[31]  Gordon J. Pace,et al.  Using gherkin to extract tests and monitors for safer medical device interaction design , 2016, EICS.

[32]  Yliès Falcone,et al.  A taxonomy for classifying runtime verification tools , 2018, International Journal on Software Tools for Technology Transfer.

[33]  Grigore Rosu,et al.  Towards Monitoring-Oriented Programming: A Paradigm Combining Specification and Implementation , 2003, RV@CAV.

[34]  Doron A. Peled,et al.  Runtime Verification: From Propositional to First-Order Temporal Logic , 2018, RV.

[35]  Grigore Rosu,et al.  Monitoring programs using rewriting , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[36]  Grigore Rosu,et al.  jPredictor , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[37]  Mahesh Viswanathan,et al.  Computational Analysis of Run-time Monitoring - Fundamentals of Java-MaC , 2002, Electron. Notes Theor. Comput. Sci..

[38]  Klaus Havelund,et al.  Runtime Verification Past Experiences and Future Projections , 2019, Computing and Software Science.

[39]  Martin Leucker,et al.  Teaching Runtime Verification , 2011, RV.

[40]  Dejan Nickovic,et al.  A survey of challenges for runtime verification from advanced application domains (beyond software) , 2018, Formal Methods in System Design.

[41]  Xian Zhang,et al.  Runtime Verification with Predictive Semantics , 2012, NASA Formal Methods.

[42]  Eric Bodden,et al.  Aspect-Oriented Race Detection in Java , 2010, IEEE Transactions on Software Engineering.

[43]  Gordon J. Pace,et al.  Safe Runtime Verification of Real-Time Properties , 2009, FORMATS.

[44]  Koushik Sen,et al.  Rule-Based Runtime Verification , 2004, VMCAI.

[45]  Grigore Rosu,et al.  Monitoring Java Programs with Java PathExplorer , 2001, RV@CAV.

[46]  Yliès Falcone,et al.  What can you verify and enforce at runtime? , 2012, International Journal on Software Tools for Technology Transfer.

[47]  Alex Groce,et al.  An Entry Point for Formal Methods: Specification and Analysis of Event Logs , 2010, FMA.

[48]  Gordon J. Pace,et al.  LARVA --- Safer Monitoring of Real-Time Java Programs (Tool Paper) , 2009, 2009 Seventh IEEE International Conference on Software Engineering and Formal Methods.

[49]  Borzoo Bonakdarpour,et al.  Runtime Verification of k-Safety Hyperproperties in HyperLTL , 2016, 2016 IEEE 29th Computer Security Foundations Symposium (CSF).

[50]  Thierry Jéron,et al.  Predictive runtime enforcement , 2016, SAC.

[51]  Klaus Havelund,et al.  Aspect-oriented monitoring of C programs , 2008 .

[52]  Martin Leucker,et al.  The Good, the Bad, and the Ugly, But How Ugly Is Ugly? , 2007, RV.