The relationship between ISO/IEC 15504 process capability levels, ISO 9001 certification and organization size: An empirical study

Abstract The gradual spread in the use of ISO 9001 and ISO/IEC 15504 (also known as software process improvement and capability determination (SPICE)) has raised questions such as “At what ISO/IEC 15504 capability level would one expect an ISO 9001 certified organization's processes to be?” and “Is there any significant difference between the ISO/IEC 15504 capability levels achieved by the processes of ISO 9001 certified organizations and those of non ISO 9001 certified organizations?”. This paper provides answers to those questions as well as to the following question “Is there any significant difference in the capability levels achieved by the ISO/IEC 15504 processes of organizations with a large information technology (IT) staff and those with a small IT staff?” In order to answer these questions, we analyzed a data set including 691 process instances (PIs) taken from 70 SPICE phase 2 trial assessments performed over the two years from September 1996 to June 1998. Results show that the ISO/IEC 15504 processes of the ISO 9001 certified organizations attained capability levels of around 1–2.3 in 15504 terms. Results also show differences between the capability levels achieved by ISO 9001 certified organizations and non ISO 9001 certified organizations, as well as between organizations with a large IT staff and those with a small IT staff.

[1]  Mark C. Paulk,et al.  How ISO 9001 Compares With The CMM , 1995, IEEE Softw..

[2]  Richard Messnarz,et al.  Better Software Practice for Business Benefit: Principles and Experiences , 1999 .

[3]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[4]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[5]  Stephen A. Freitas,et al.  Modern Industrial Statistics: Design and Control of Quality and Reliability , 1999, Technometrics.

[6]  P. Good,et al.  Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses , 1995 .

[7]  Khaled El Emam,et al.  An empirical evaluation of the ISO/IEC 15504 assessment model , 2001, J. Syst. Softw..

[8]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability , 2000, IEEE Trans. Software Eng..

[9]  Dennis R. Goldenson,et al.  SPICE: an empiricist's perspective , 1995, Proceedings of Software Engineering Standards Symposium.

[10]  P. Gardner Scales and Statistics , 1975 .

[11]  C. R. Mehta,et al.  StatXact : a statistical package for exact non-parametric inference, Cytel Software Corporation, Cambridge, MA, USA , 1990 .

[12]  Khaled El Emam,et al.  Spice: The Theory and Practice of Software Process Improvement and Capability Determination , 1997 .

[13]  Suzanne Garcia,et al.  Key Practices of the Capability Maturity Model, Version 1.1 , 1993 .

[14]  S. S. Stevens,et al.  Handbook of experimental psychology , 1951 .

[15]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 measures of software development process capability , 2000, J. Syst. Softw..

[16]  Leonard R. Sussman,et al.  Nominal, Ordinal, Interval, and Ratio Typologies are Misleading , 1993 .

[17]  Norman Fenton,et al.  Towards the evaluation of software engineering standards , 1993, Proceedings 1993 Software Engineering Standards Symposium.

[18]  Karen D. Prenger,et al.  Costs and Benefits of Software Process Improvement , 1997 .

[19]  S. S. Stevens Mathematics, measurement, and psychophysics. , 1951 .

[20]  Richard H. Thayer,et al.  Software Engineering: A European Perspective , 1993 .

[21]  A. Agresti An introduction to categorical data analysis , 1997 .

[22]  Shari Lawrence Pfleeger,et al.  Evaluating software engineering standards , 1994, Computer.