The Internal Consistency of Key Process Areas in the Capability Maturity Model (CMM) for Software (SW-CMM)

Abstract : Evaluating the reliability of maturity level ratings is crucial for providing confidence in the results of software process assessments. This report examines the dimensions underlying the maturity construct in the Capability Maturity Model (CMM) for Software (SW-CMM) and then estimates the internal consistency (reliability) of each dimension. The analysis is based on 676 CMM-Based Appraisal for Internal Process Improvement (CBA IPI) assessments conducted during the period of January 2000 through April 2002. The results suggest that the SW-CMM maturity is a three-dimensional construct, with 'Project Implementation' representing the maturity level 2 key process areas (KPAs), 'Organization Implementation' representing the maturity level 3 KPAs, and 'Quantitative Process Management' representing the KPAs at both maturity levels 4 and 5. The internal consistency for each of the three dimensions as estimated by Cronbach's alpha exceeds the recommended value of 0.9. Although more should be learned about the distinctions between maturity levels 4 and 5, the internal consistency of those KPAs is comparable to those at levels 2 and 3.

[1]  Barry Boehm,et al.  The effects of software process maturity on software development effort , 1997 .

[2]  Edward G. Carmines,et al.  Measurement in the Social Sciences: The Link Between Theory and Data , 1980, American Political Science Review.

[3]  A. Lederer,et al.  An Instrument for Assessing the Organizational Benefits of IS Projects , 1998 .

[4]  Ananth Srinivasan,et al.  Alternative Measures of Systems Effectiveness: Associations and Implications , 1985, MIS Q..

[5]  Pasi Kuvaja.  BOOTSTRAP 3.0—A SPICE1 Conformant Software Process Assessment Methodology , 2004, Software Quality Journal.

[6]  Khaled El Emam,et al.  Benchmarking Kappa: Interrater Agreement in Software Process Assessments , 1999, Empirical Software Engineering.

[7]  Kwasi Amoako-Gyampah,et al.  User involvement and user satisfaction: An exploratory contingency model , 1993, Inf. Manag..

[8]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[9]  Vijay Sethi,et al.  Construct Measurement in Information Systems Research: An Illustration in Strategic Systems , 1991 .

[10]  Mark C. Paulk,et al.  The Evolation of the SEI's Capacity Maturity Model for Software , 1995, Softw. Process. Improv. Pract..

[11]  Mark C. Paulk,et al.  Key Practices of the Capability Maturity Model , 1991 .

[12]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability , 2000, IEEE Trans. Software Eng..

[13]  Tore Dybå,et al.  An Instrument for Measuring the Key Factors of Success in Software Process Improvement , 2000, Empirical Software Engineering.

[14]  Bob Smith,et al.  The Internal Consistencies of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension , 1998, Empirical Software Engineering.

[15]  Leon A. Kappelman,et al.  Cautions on the Use of the SERVQUAL Measure to Assess the Quality of Information Systems Services , 1999 .

[16]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[17]  C. Lunneborg.  Data Analysis by Resampling: Concepts and Applications , 1999 .

[18]  Mauri Laitinen,et al.  Process assessment considered wasteful , 1997, CACM.

[19]  Donna K. Dunaway,et al.  CMM-Based Appraisal for Internal Process Improvement (CBA IPI): Method Description , 1996 .

[20]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[21]  Dale L. Goodhue,et al.  Development and Measurement Validity of a Task-Technology Fit Instrument for User Evaluations of Inf , 1998 .

[22]  Hossein Saiedian,et al.  SEI Capability Maturity Model's Impact on Contractors , 1995, Computer.

[23]  Subhash Sharma.  Applied multivariate techniques , 1995 .

[24]  Paul E. Spector.  Summated Rating Scale Construction: An Introduction , 1992 .

[25]  Mark C. Paulk,et al.  Capability Maturity Model for Software, Version 1.1 , 1993 .

[26]  Iris Vessey,et al.  The Effect of User Involvement on System Success: A Contingency Approach , 1988, MIS Q..

[27]  Khaled El Emam,et al.  Estimating the extent of standards use: the case of ISO/IEC 15504 , 2000, J. Syst. Softw..

[28]  Angela Tuffley,et al.  Process capability in the Australian software industry-results from the SPICE trials , 1998, Proceedings 1998 Australian Software Engineering Conference (Cat. No.98EX233).

[29]  H. Kaiser.  A second generation little jiffy , 1970 .

[30]  Watts S. Humphrey,et al.  A method for assessing the software engineering capability of contractors , 1987 .

[31]  Khaled El Emam,et al.  Findings from Phase 2 of the SPICE trials , 2001, Softw. Process. Improv. Pract..

[32]  Sree Nilakanta,et al.  Measurement: A blueprint for theory-building in MIS , 1994, Inf. Manag..

[33]  Norman Cliff,et al.  The eigenvalues-greater-than-one rule and the reliability of components. , 1988 .

[34]  Watts S. Humphrey,et al.  Conducting SEI-Assisted Software Process Assessments , 1989 .

[35]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[36]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 measures of software development process capability , 2000, J. Syst. Softw..

[37]  James D. Herbsleb,et al.  Software quality and the Capability Maturity Model , 1997, CACM.

[38]  Eric R. Ziegel,et al.  Engineering Statistics , 2004, Technometrics.

[39]  B. I. Blum,et al.  Characterizing the software process , 1994 .

[40]  Mark C. Paulk,et al.  Analyzing the Conceptual Relationship Between ISO/IEC 15504 (Software Process Assessment) and the Capability Maturity Model for Software , 1999 .

[41]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[42]  Ho-Won Jung,et al.  The relationship between ISO/IEC 15504 process capability levels, ISO 9001 certification and organization size: An empirical study , 2001, J. Syst. Softw..

[43]  Khaled El Emam,et al.  Cost implications of interrater agreement for software process assessments , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[44]  Mark C. Paulk,et al.  Capability Maturity Model , 1991 .

[45]  Watts S. Humphrey,et al.  Characterizing the software process: a maturity framework , 1988, IEEE Software.

[46]  Dennis F. Galletta,et al.  Some Cautions on the Measurement of User Information Satisfaction , 1989 .

[47]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[48]  James D. Herbsleb,et al.  Modelling the Likelihood of Software Process Improvement: An Exploratory Study , 2004, Empirical Software Engineering.

[49]  A. V. D. Ven,et al.  Measuring And Assessing Organizations , 1980 .

[50]  R. Lissitz,et al.  Effect of the number of scale points on reliability: A Monte Carlo approach. , 1975 .

[51]  John Gould,et al.  8 Project implementation , 1999 .

[52]  Matthew J. Fisher,et al.  Software Acquisition Capability Maturity Model (SA-CMM) Version 1.02 , 1999 .

[53]  Mike Phillips,et al.  Software Capability Evaluation Version 3.0 Method Description , 1996 .

[54]  Mark C. Paulk,et al.  The Capability Maturity Model: Guidelines for Improving the Software Process , 1994 .

[55]  Jack Ferguson.  Software Acquisition Capability Maturity Model (SA-CMM) , 2002 .

[56]  Suzanne Garcia,et al.  Key Practices of the Capability Maturity Model, Version 1.1 , 1993 .

[57]  Khaled El Emam,et al.  An empirical review of software process assessments , 2000, Adv. Comput..

[58]  Khaled El Emam,et al.  The Internal Consistency of the ISO/IEC 15504 Software Process Capability Scale , 1998, IEEE METRICS.

[59]  Peter A. Todd,et al.  Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication , 1992, MIS Q..

[60]  William J. Doll,et al.  The test-retest reliability of user involvement instruments , 1994, Inf. Manag..

[61]  W. L. Smith,et al.  On the limitations of software process assessment and the recognition of a required re-orientation for global process improvement , 1998, Software Quality Journal.

[62]  Mayuram S. Krishnan,et al.  Measuring Process Consistency: Implications for Reducing Software Defects , 1999, IEEE Trans. Software Eng..

[63]  Stephen M. Masters,et al.  CMM Appraisal Framework, Version 1.0 , 1995 .