Formal Specification-Based Inspection for Verification of Programs

Software inspection is a static analysis technique that is widely used for defect detection, but which suffers from a lack of rigor. In this paper, we address this problem by taking advantage of formal specification and analysis to support a systematic and rigorous inspection method. The aim of the method is to use inspection to determine whether every functional scenario defined in the specification is implemented correctly by a set of program paths and whether every program path of the program contributes to the implementation of some functional scenario in the specification. The method is comprised of five steps: deriving functional scenarios from the specification, deriving paths from the program, linking scenarios to paths, analyzing paths against the corresponding scenarios, and producing an inspection report, and allows for a systematic and automatic generation of a checklist for inspection. We present an example to show how the method can be used, and describe an experiment to evaluate its performance by comparing it to perspective-based reading (PBR). The result shows that our method may be more effective in detecting function-related defects than PBR but slightly less effective in detecting implementation-related defects. We also describe a prototype tool to demonstrate the supportability of the method, and draw some conclusions about our work.

[1]  Harvey P. Siy,et al.  An experiment to assess the cost-benefits of code inspections in large scale software development , 1995, SIGSOFT '95.

[2]  Taro Kurita,et al.  Application of a Formal Specification Language in the Development of the "Mobile FeliCa" IC Chip Firmware for Embedding in Mobile Phone , 2008, FM.

[3]  Harvey P. Siy,et al.  An Experiment ot Assess the Cost-Benefits of Code Inspections in Large Scale Software Development , 1997, IEEE Trans. Software Eng..

[4]  David Lorge Parnas,et al.  The Role of Inspection in Software Quality Assurance , 2003, IEEE Trans. Software Eng..

[5]  Forrest Shull,et al.  The empirical investigation of Perspective-Based Reading , 1995, Empirical Software Engineering.

[6]  Susan Stepney A tale of two proofs , 1998, FM-Trends 1998.

[7]  Christel Baier,et al.  PROBMELA: a modeling language for communicating probabilistic processes , 2004, Proceedings. Second ACM and IEEE International Conference on Formal Methods and Models for Co-Design, 2004. MEMOCODE '04..

[8]  Forrest Shull,et al.  Detecting defects in object-oriented designs: using reading techniques to increase software quality , 1999, OOPSLA '99.

[9]  Tatsuhiro Tsuchiya,et al.  On fault classes and error detection capability of specification-based testing , 2002, TSEM.

[10]  Marsha Chechik,et al.  Automatic Analysis of Consistency between Requirements and Designs , 2001, IEEE Trans. Software Eng..

[11]  David Lorge Parnas,et al.  Precise Documentation of Well-Structured Programs , 1994, IEEE Trans. Software Eng..

[12]  Takeo Kanade,et al.  Unifying Theories of Programming , 2010, Lecture Notes in Computer Science.

[13]  O. Laitenberger A Survey of Software Inspection Technologies , 2001 .

[14]  William R. Bush,et al.  A static analyzer for finding dynamic programming errors , 2000, Softw. Pract. Exp..

[15]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[16]  Harvey P. Siy,et al.  A Review of Software Inspections , 1995, Adv. Comput..

[17]  Alan Wassyng,et al.  Software tools for safety-critical software development , 2006, International Journal on Software Tools for Technology Transfer.

[18]  Arthur L. Price,et al.  Managing code inspection information , 1994, IEEE Software.

[19]  Gilbert Babin,et al.  Application of Formal Methods to Scenario-Based Requirements Engineering , 2001 .

[20]  C. A. R. Hoare,et al.  An Axiomatic Definition of the Programming Language PASCAL , 1973, Acta Informatica.

[21]  Richard A. Kemmerer,et al.  Testing Formal Specifications to Detect Design Errors , 1985, IEEE Transactions on Software Engineering.

[22]  John C. Knight,et al.  Why are Formal Methods Not Used More Widely , 1997 .

[23]  Marie-Claude Gaudel,et al.  Software testing based on formal specifications: a theory and a tool , 1991, Softw. Eng. J..

[24]  Tim Miller,et al.  Model-Based Specification Animation Using Testgraphs , 2002, ICFEM.

[25]  David A. Carrington,et al.  A Framework for Specification-Based Testing , 1996, IEEE Trans. Software Eng..

[26]  Sami Kollanus,et al.  Survey of Software Inspection Research: 1991-2005 , 2007 .

[27]  Constance L. Heitmeyer,et al.  Software Cost Reduction , 2002 .

[28]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[29]  Shaoying Liu,et al.  An Automated Approach to Specification-Based Program Inspection , 2005, ICFEM.

[30]  Andreas Reuys,et al.  Derivation of Domain Test Scenarios from Activity Diagrams1 , 2003 .

[31]  D. Richard Kuhn Fault classes and error detection capability of specification-based testing , 1999, TSEM.

[32]  Michael E. Fagan Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[33]  Christel Baier,et al.  Principles of model checking , 2008 .

[34]  Stephen A. Stelting,et al.  Applied Java Patterns , 2002 .

[35]  Khaled El Emam,et al.  An Internally Replicated Quasi-Experimental Comparison of Checklist and Perspective-Based Reading of Code Documents , 2001, IEEE Trans. Software Eng..

[36]  Marc Roper,et al.  Practical Code Inspection Techniques for Object-Oriented Systems: An Experimental Comparison , 2003, IEEE Softw..

[37]  Victor R. Basili,et al.  Comparing the Effectiveness of Software Testing Strategies , 1987, IEEE Transactions on Software Engineering.

[38]  Jawed I. A. Siddiqi,et al.  A toolset to support the construction and animation of formal specifications , 1998, J. Syst. Softw..

[39]  Diane Kelly,et al.  Task-directed software inspection technique: an experiment and case study , 2000, CASCON.

[40]  Claes Wohlin,et al.  An Experimental Comparison of Usage-Based and Checklist-Based Reading , 2003, IEEE Trans. Software Eng..

[41]  Shaoying Liu Formal Engineering for Industrial Software Development: Using the SOFL Method , 2004 .

[42]  Jean-Raymond Abrial,et al.  The B-book - assigning programs to meanings , 1996 .

[43]  Jin Xiao-Hui,et al.  Use of tabular expressions in the inspection of concurrent programs , 2005 .

[44]  David Lorge Parnas,et al.  Active design reviews: principles and practices , 1985, ICSE '85.

[45]  Alan Wassyng,et al.  Lessons Learned from a Successful Implementation of Formal Methods in an Industrial Project , 2003, FME.

[46]  Shaoying Liu,et al.  Formal Engineering for Industrial Software Development , 2004, Springer Berlin Heidelberg.

[47]  John Dawes,et al.  The VDM-SL Reference Guide , 1991 .

[48]  Shaoying Liu,et al.  An approach to specifying and verifying safety-critical systems with practical formal method SOFL , 1998, Proceedings. Fourth IEEE International Conference on Engineering of Complex Computer Systems (Cat. No.98EX193).

[49]  Shaoying Liu,et al.  A relation-based method combining functional and structural testing for test case generation , 2008, J. Syst. Softw..

[50]  Per Runeson,et al.  Usage-based readingan experiment to guide reviewers with use cases , 2001, Inf. Softw. Technol..

[51]  Claes Wohlin,et al.  State‐of‐the‐art: software inspections after 25 years , 2002, Softw. Test. Verification Reliab..

[52]  Carroll Morgan,et al.  Programming from specifications , 1990, Prentice Hall International Series in computer science.

[53]  Jeremy Dick,et al.  Automating the Generation and Sequencing of Test Cases from Model-Based Specifications , 1993, FME.

[54]  Peter Gorm Larsen,et al.  Applying Formal Specification in Industry , 1996, IEEE Softw..

[55]  Karl E. Wiegers,et al.  Peer Reviews in Software: A Practical Guide , 2001 .

[56]  Ralph-Johan Back,et al.  Refinement Calculus: A Systematic Introduction , 1998 .

[57]  Leon J. Osterweil,et al.  On Two Problems in the Generation of Program Test Paths , 1976, IEEE Transactions on Software Engineering.

[58]  Anneke Kleppe,et al.  The Object Constraint Language: Getting Your Models Ready for MDA , 2003 .

[59]  Edsger W. Dijkstra,et al.  Predicate Calculus and Program Semantics , 1989, Texts and Monographs in Computer Science.

[60]  Pankaj Jalote,et al.  ON OPTIMUM MODULE SIZE FOR SOFTWARE INSPECTIONS , 2007 .

[61]  David Lorge Parnas,et al.  Assessment of safety-critical software in nuclear power plants , 1991 .

[62]  James R. Lyle,et al.  A Two-Person Inspection Method to Improve Prog ramming Productivity , 1989, IEEE Transactions on Software Engineering.

[63]  Bill Brykczynski,et al.  Software inspection : an industry best practice , 1996 .

[64]  David Lorge Parnas,et al.  Tabular Representation of Relations , 1992 .

[65]  John C. Knight,et al.  An improved inspection technique , 1993, CACM.

[66]  Jim Woodcock,et al.  Using Z - specification, refinement, and proof , 1996, Prentice Hall international series in computer science.

[67]  Anthony Hall,et al.  Realising the Benefits of Formal Methods , 2005, J. Univers. Comput. Sci..

[68]  B BisantDavid,et al.  A Two-Person Inspection Method to Improve Programming Productivity , 1989 .

[69]  Daniel Jackson,et al.  Alloy: a lightweight object modelling notation , 2002, TSEM.

[70]  Daniel Jackson,et al.  Alcoa: the Alloy constraint analyzer , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[71]  Jason McDonald,et al.  Translating Object-Z specifications to passive test oracles , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[72]  Constance L. Heitmeyer,et al.  Applying the SCR requirements method to a weapons control panel: an experience report , 1998, FMSP '98.