Investigating Safety of a Radiotherapy Machine Using System Models with Pluggable Checkers

Formal techniques for guaranteeing software correctness have made tremendous progress in recent decades. However, applying these techniques to real-world safety-critical systems remains challenging in practice. Inspired by goals set out in prior work, we report on a large-scale case study that applies modern verification techniques to check safety properties of a radiotherapy system in current clinical use. Because of the diversity and complexity of the system’s components (software, hardware, and physical), no single tool was suitable for both checking critical component properties and ensuring that their composition implies critical system properties. This paper describes how we used state-of-the-art approaches to develop specialized tools for verifying safety properties of individual components, as well as an extensible tool for composing those properties to check the safety of the system as a whole. We describe the key design decisions that diverged from previous approaches and that enabled us to practically apply our approach to provide machine-checked guarantees. Our case study uncovered subtle safety-critical flaws in a pre-release of the latest version of the radiotherapy system’s control software.

[1]  J.P.Jacky EPICS-based Control System for a Radiation Therapy Machine , 2013 .

[2]  Alan J. Hu,et al.  Calysto: scalable and precise extended static checking , 2008, ICSE.

[3]  John Rushby,et al.  Mechanized Support for Assurance Case Argumentation , 2013, JSAI-isAI Workshops.

[4]  Joseph P. Near,et al.  A lightweight code analysis and its role in evaluation of a dependability case , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[5]  Jonathan Jacky,et al.  Clinical Neutron Therapy System Therapist's Guide , 2002 .

[6]  Frank Tip,et al.  Finding bugs efficiently with a SAT solver , 2007, ESEC-FSE '07.

[7]  Jonathan Jacky,et al.  A Control System for a Radiation Therapy Machine , 2001 .

[8]  Daniel Kroening,et al.  A Tool for Checking ANSI-C Programs , 2004, TACAS.

[9]  Daniel Jackson,et al.  A direct path to dependable software , 2009, CACM.

[10]  Jonathan Jacky Formal Safety Analysis of the Control Program for a Radiation Therapy Machine , 2000 .

[11]  John C. Knight,et al.  A Taxonomy of Fallacies in System Safety Arguments , 2006 .

[12]  Gerard J. Holzmann,et al.  Mars code , 2014, CACM.

[13]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[14]  Daniel Jackson,et al.  Property-part diagrams: A dependence notation for software systems , 2009 .

[15]  Tim Kelly,et al.  The Goal Structuring Notation – A Safety Argument Notation , 2004 .

[16]  Konrad Slind,et al.  Resolute: an assurance case language for architecture models , 2014 .

[17]  Alexander Aiken,et al.  Saturn: A scalable framework for error detection using Boolean satisfiability , 2007, TOPL.

[18]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[19]  Lawrence Charles Paulson,et al.  Isabelle/HOL: A Proof Assistant for Higher-Order Logic , 2002 .

[20]  Emina Torlak,et al.  Kodkod: A Relational Model Finder , 2007, TACAS.

[21]  Emina Torlak,et al.  Growing solver-aided languages with rosette , 2013, Onward!.

[22]  Ewen Denney,et al.  AdvoCATE: An Assurance Case Automation Toolset , 2012, SAFECOMP Workshops.

[23]  Martyn Thomas,et al.  Software for Dependable Systems: Sufficient Evidence? , 2007 .

[24]  Emina Torlak,et al.  A lightweight symbolic virtual machine for solver-aided host languages , 2014, PLDI.

[25]  Natarajan Shankar,et al.  PVS: A Prototype Verification System , 1992, CADE.

[26]  Emina Torlak,et al.  A constraint solver for software engineering: finding models and cores of large relational specifications , 2009 .

[27]  Gerard J. Holzmann,et al.  The power of 10: rules for developing safety-critical code , 2006, Computer.

[28]  John M. Rushby,et al.  Formalism in Safety Cases , 2010, SSS.

[29]  Xi Wang,et al.  Toward a Dependability Case Language and Workflow for a Radiation Therapy System , 2015, SNAPL.

[30]  Srinath T. V. Setty,et al.  IronFleet: proving practical distributed systems correct , 2015, SOSP.

[31]  Jonathan Jacky,et al.  Clinical Neutron Therapy System Reference Manual , 2002 .

[32]  Natarajan Shankar,et al.  Tool Integration with the Evidential Tool Bus , 2013, VMCAI.

[33]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[34]  Andrew W. Appel,et al.  Verified Correctness and Security of OpenSSL HMAC , 2015, USENIX Security Symposium.

[35]  Ewen Denney,et al.  Evidence arguments for using formal methods in software certification , 2013, 2013 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).