Formal Methods: From Academia to Industrial Practice. A Travel Guide

For many decades, formal methods are considered to be the way forward to help the software industry to make more reliable and trustworthy software. However, despite this strong belief and many individual success stories, no real change in industrial software development seems to be occurring. In fact, the software industry itself is moving forward rapidly, and the gap between what formal methods can achieve and the daily software-development practice does not appear to be getting smaller (and might even be growing). In the past, many recommendations have already been made on how to develop formal-methods research in order to close this gap. This paper investigates why the gap nevertheless still exists and provides its own recommendations on what can be done by the formal-methods-research community to bridge it. Our recommendations do not focus on open research questions. In fact, formal-methods tools and techniques are already of high quality and can address many non-trivial problems; we do give some technical recommendations on how tools and techniques can be made more accessible. To a greater extent, we focus on the human aspect: how to achieve impact, how to change the way of thinking of the various stakeholders about this issue, and in particular, as a research community, how to alter our behaviour, and instead of competing, collaborate to address this issue.

[1]  David R. Cok,et al.  Java Automated Deductive Verification in Practice: Lessons from Industrial Proof-Based Projects , 2018, ISoLA.

[2]  Frits W. Vaandrager,et al.  Lectures on Embedded Systems, European Educational Forum, School on Embedded Systems , 1996 .

[3]  Michael Norrish,et al.  seL4: formal verification of an operating-system kernel , 2010, Commun. ACM.

[4]  Antoine Miné Static Analysis of Embedded Real-Time Concurrent Software with Dynamic Priorities , 2017, Electron. Notes Theor. Comput. Sci..

[5]  James J. Hunt,et al.  Java in Safety Critical Systems , 2010 .

[6]  Paris Avgeriou,et al.  Stories from the Front , 2018 .

[7]  Jan Friso Groote,et al.  Software engineering: Redundancy is key , 2015, Sci. Comput. Program..

[8]  Hardi Hungar,et al.  Model Generation by Moderated Regular Extrapolation , 2002, FASE.

[9]  Andrew Peter Wallace McCarthy E DITOR ’ S C OMMENTS Diversity of Design Science Research , 2022 .

[10]  Michael Felderer,et al.  Formal Methods in Industrial Practice - Bridging the Gap (Track Summary) , 2018, ISoLA.

[11]  Tiziana Margaria,et al.  Tools get formal methods into practice , 1996, CSUR.

[12]  Tiziana Margaria,et al.  Foundations of Semantics and Model Checking in a Software Engineering Course , 2015, FMSEE&T@FM.

[13]  Virginie Wiels,et al.  Formal Verification of Avionics Software Products , 2009, FM.

[14]  Tiziana Margaria,et al.  METAFrame in Practice: Design of Intelligent Network Services , 1999, Correct System Design.

[15]  Pim Kars The application of Promela and Spin in the BOS project , 1996, The Spin Verification System.

[16]  Tiziana Margaria,et al.  A Summer Computing Camp Using ChainReaction and jABC , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[17]  Fan Zhang,et al.  Use of Formal Methods at Amazon Web Services , 2014 .

[18]  Wan Fokkink,et al.  Some Trends in Formal Methods Applications to Railway Signaling , 2012, FMICS 2012.

[19]  Frank S. de Boer,et al.  OpenJDK's Java.utils.Collection.sort() Is Broken: The Good, the Bad and the Worst Case , 2015, CAV.

[20]  Thomas Ball,et al.  SLAM2: Static driver verification with under 4% false alarms , 2010, Formal Methods in Computer Aided Design.

[21]  Catherine A. Meadows,et al.  Emerging Issues and Trends in Formal Methods in Cryptographic Protocol Analysis: Twelve Years Later , 2015, Logic, Rewriting, and Concurrency.

[22]  Judy Bowen,et al.  The Handbook of Formal Methods in Human-Computer Interaction , 2017, Human–Computer Interaction Series.

[23]  Tiziana Margaria,et al.  Formal Methods for Industrial Critical Systems: A Survey of Applications , 2012 .

[24]  Gavin Lowe,et al.  Breaking and Fixing the Needham-Schroeder Public-Key Protocol Using FDR , 1996, Softw. Concepts Tools.

[25]  Tiziana Margaria,et al.  jETI: A Tool for Remote Tool Integration , 2005, TACAS.

[26]  Alessandro Fantechi,et al.  A Story About Formal Methods Adoption by a Railway Signaling Manufacturer , 2006, FM.

[27]  Jozef Hooman,et al.  Evaluating the effect of a lightweight formal technique in industry , 2016, International Journal on Software Tools for Technology Transfer.

[28]  Dan Craigen,et al.  An International Survey of Industrial Applications of Formal Methods , 1992, Z User Workshop.

[29]  Marijn J. H. Heule,et al.  Proceedings of SAT Competition 2017: Solver and Benchmark Descriptions , 2017 .

[30]  Jeannine Siviy,et al.  Using the Technology Readiness Levels Scale to Support Technology Management in the DoD's ATD/STO Environments (A Findings and Recommendations Report Conducted for Army CECOM) , 2002 .

[31]  Hans Knutsson,et al.  Correction for Eklund et al., Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[32]  Jan Friso Groote,et al.  Experience Report on Designing and Developing Control Components Using Formal Methods , 2012, FM.

[33]  Thomas Ball,et al.  CloudSDV Enabling Static Driver Verifier Using Microsoft Azure , 2016, IFM.

[34]  Xavier Leroy,et al.  The CompCert Memory Model, Version 2 , 2012 .

[35]  Patrick Behm,et al.  MÉTÉOR : An Industrial Success in Formal Development , 1998, B.

[36]  Anthony Hall,et al.  Seven myths of formal methods , 1990, IEEE Software.

[37]  Jonathan P. Bowen,et al.  Formal Methods , 2010, Computing Handbook, 3rd ed..

[38]  Jean-Christophe Filliâtre,et al.  VerifyThis 2015 A program verification competition , 2016 .

[39]  Michiel van Genuchten,et al.  Metrics with Impact , 2013, IEEE Software.

[40]  Edmund M. Clarke,et al.  Formal Methods: State of the Art and Future Directions Working Group Members , 1996 .

[41]  Patrick Cousot,et al.  The ASTREÉ Analyzer , 2005, ESOP.

[42]  Jean-Louis Boulanger,et al.  Formal Methods: Industrial Use from Model to the Code , 2013 .

[43]  Michel R. V. Chaudron,et al.  Software Engineering with Formal Methods: The Development of a Storm Surge Barrier Control System Revisiting Seven Myths of Formal Methods , 2001, Formal Methods Syst. Des..

[44]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[45]  Archana Ganapathi,et al.  Crash data collection: a Windows case study , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[46]  Peter W. O'Hearn,et al.  Moving Fast with Software Verification , 2015, NFM.

[47]  Erik Poll,et al.  VerifyThis 2016: A Program Verification Competition , 2016 .

[48]  Christophe Ponsard,et al.  High-Level Guidance for Managers Deploying Formal Methods in Their Organisation , 2013, FMICS.

[49]  David S. Johnson,et al.  Dimacs series in discrete mathematics and theoretical computer science , 1996 .

[50]  J. Srinivas,et al.  Scrum Software Development , 2016 .

[51]  Taylor Walsh,et al.  Unlocking the Gates: How and Why Leading Universities Are Opening Up Access to Their Courses , 2010 .

[52]  Yannick Moy,et al.  Testing or Formal Verification: DO-178C Alternatives and Industrial Experience , 2013, IEEE Software.

[53]  Tiziana Margaria,et al.  DSLs for Decision Services: A Tutorial Introduction to Language-Driven Engineering , 2018, ISoLA.

[54]  Mark A. Hillebrand,et al.  Verifying C Programs : A VCC Tutorial Working draft , version 0 . 2 , April 18 , 2011 , 2011 .

[55]  Nikolai Kosmatov,et al.  Frama-C: A software analysis perspective , 2015, Formal Aspects of Computing.

[56]  Bernhard Steffen,et al.  The RERS 2017 challenge and workshop (invited paper) , 2017, SPIN.

[57]  Bernhard Steffen,et al.  CINCO: a simplicity-driven approach to full generation of domain-specific graphical modeling tools , 2018, International Journal on Software Tools for Technology Transfer.

[58]  Pim Kars,et al.  Formal Methods in the Design of s Storm Surge Barrier Control System , 1996, European Educational Forum: School on Embedded Systems.

[59]  Les Hatton The Chimera of Software Quality , 2007, Computer.

[60]  Daniel M. Zimmerman,et al.  Secret Ninja Formal Methods , 2008, FM.