An Experimental Comparison of Usage-Based and Checklist-Based Reading

Software quality can be defined as the customers' perception of how a system works. Inspection is a method to monitor and control the quality throughout the development cycle. Reading techniques applied to inspections help reviewers to stay focused on the important parts of an artifact when inspecting. However, many reading techniques focus on finding as many faults as possible, regardless of their importance. Usage-based reading helps reviewers to focus on the most important parts of a software artifact from a user's point of view. We present an experiment, which compares usage-based and checklist-based reading. The results show that reviewers applying usage-based reading are more efficient and effective in detecting the most critical faults from a user's point of view than reviewers using checklist-based reading. Usage-based reading may be preferable for software organizations that utilize or start utilizing use cases in their software development.

[1]  John D. Musa,et al.  Software reliability engineering : more reliable software, faster development and testing , 1999 .

[2]  Stephen G. Eick,et al.  Estimating software fault content before coding , 1992, International Conference on Software Engineering.

[3]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[4]  John C. Knight,et al.  An improved inspection technique , 1993, CACM.

[5]  Forrest Shull,et al.  Detecting defects in object-oriented designs: using reading techniques to increase software quality , 1999, OOPSLA '99.

[6]  Colin Atkinson,et al.  An experimental comparison of reading techniques for defect detection in UML design documents , 2000, J. Syst. Softw..

[7]  R.C. Linger,et al.  Cleanroom process model , 1994, IEEE Software.

[8]  James R. Lyle,et al.  A Two-Person Inspection Method to Improve Prog ramming Productivity , 1989, IEEE Transactions on Software Engineering.

[9]  Per Runeson,et al.  Are the Perspectives Really Different? – Further Experimentation on Scenario-Based Reading of Requirements , 2000, Empirical Software Engineering.

[10]  Adam A. Porter,et al.  Comparing Detection Methods For Software Requirements Inspections: A Replication Using Professional Subjects , 1998, Empirical Software Engineering.

[11]  Claes Wohlin,et al.  How much information is needed for usage-based reading? A series of experiments , 2002, Proceedings International Symposium on Empirical Software Engineering.

[12]  Giuseppe Visaggio,et al.  A Replicated Experiment to Assess Requirements Inspection Techniques , 2004, Empirical Software Engineering.

[13]  S. Lauesen.  Software Requirements Styles and Techniques , 2001 .

[14]  Forrest Shull,et al.  How perspective-based reading can improve requirements inspections , 2000, Computer.

[15]  Forrest Shull,et al.  Developing techniques for using software documents: a series of empirical studies , 1998 .

[16]  Forrest Shull,et al.  The empirical investigation of Perspective-Based Reading , 1995, Empirical Software Engineering.

[17]  Marc Roper,et al.  Software inspections , 2003, Adv. Comput..

[18]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[19]  James Miller,et al.  Further Experiences with Scenarios and Checklists , 1998, Empirical Software Engineering.

[20]  Edward F. Weller,et al.  Lessons from three years of inspection data (software development) , 1993, IEEE Software.

[21]  David Lorge Parnas,et al.  Active design reviews: principles and practices , 1985, ICSE '85.

[22]  Giuseppe Visaggio,et al.  Evaluating Defect Detection Techniques for Software Requirements Inspections , 2000 .

[23]  Oliver Laitenberger,et al.  An encompassing life cycle centric survey of software inspection , 2000, J. Syst. Softw..

[24]  Ola Blomkvist,et al.  An Extended Replication of an Experiment for Assessing Methods for Software Requirements Inspections , 1998, Empirical Software Engineering.

[25]  Claes Wohlin,et al.  Statistical Usage Testing for Software Reliability Control , 1995, Informatica.

[26]  Ben Shneiderman,et al.  Perspective-based Usability Inspection: An Empirical Validation of Efficacy , 1999, Empirical Software Engineering.

[27]  Will Hayes,et al.  Research synthesis in software engineering: a case for meta-analysis , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[28]  B BisantDavid,et al.  A Two-Person Inspection Method to Improve Programming Productivity , 1989 .

[29]  Wei-Tek Tsai,et al.  N-Fold inspection: a requirements analysis technique , 1990, Commun. ACM.

[30]  Robert G. Ebenau,et al.  Software Inspection Process , 1993 .

[31]  Stefan Biffl,et al.  Using reading techniques to focus inspection performance , 2001, Proceedings 27th EUROMICRO Conference. 2001: A Net Odyssey.

[32]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[33]  Adam A. Porter,et al.  An experiment to assess different defect detection methods for software requirements inspections , 1994, Proceedings of 16th International Conference on Software Engineering.

[34]  Itu-T.  Specification and Description Language (SDL) , 1999 .

[35]  Sivert Sørumgård,et al.  Verification of Process Conformance in Empirical Studies of Software Development , 1997 .

[36]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[37]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[38]  John D. Musa,et al.  Software Reliability Engineering: More Reliable Software Faster and Cheaper , 2004 .

[39]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[40]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[41]  Oliver Laitenberger,et al.  Perspective-based reading of code documents at Robert Bosch GmbH , 1997, Inf. Softw. Technol..

[42]  Colin Atkinson,et al.  Generalizing perspective-based inspection to handle object-oriented development artifacts , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[43]  Soren Lauesen,et al.  Software Requirements: Styles & Techniques , 2002 .

[44]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[45]  Joachim Karlsson,et al.  A Cost-Value Approach for Prioritizing Requirements , 1997, IEEE Softw..

[46]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[47]  Ivar Jacobson,et al.  Object-oriented software engineering - a use case driven approach , 1993, TOOLS.

[48]  Per Runeson,et al.  Usage-based readingan experiment to guide reviewers with use cases , 2001, Inf. Softw. Technol..