The Internal Consistencies of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension

This paper presents the results of an empirical evaluation of the reliability of two commonly used assessment instruments: the 1987 SEI Maturity Questionnaire and the SPICE v1 capability dimension. The type of reliability that was evaluated is internal consistency. A study of the internal consistency of the 1987 questionnaire was only briefly mentioned in a 1991 article, and the internal consistency of the SPICE v1 capability dimension has not been evaluated thus far. We used two different data sets to evaluate the internal consistency of each instrument. Our results indicate that both assessment instruments are very reliable and also have similar reliability levels. The results are encouraging for users of assessment instruments, and provide a baseline with which to compare subsequent versions of these instruments.

[1]  Deborah A. Boehm-Davis,et al.  Empirical Studies of Programmers: Sixth Workshop , 1996 .

[2]  M. Mckubre.  The Importance of Replication , 2007 .

[3]  Watts S. Humphrey,et al.  A method for assessing the software engineering capability of contractors , 1987 .

[4]  Hossein Saiedian,et al.  SEI Capability Maturity Model's Impact on Contractors , 1995, Computer.

[5]  David Rugg.  Using a capability evaluation to select a contractor , 1993, IEEE Software.

[6]  Terry Bollinger,et al.  A critical look at software capability evaluations , 1991, IEEE Software.

[7]  S. Dekleva,et al.  Delphi study of software maintenance problems , 1992, Proceedings Conference on Software Maintenance 1992.

[8]  Watts S. Humphrey,et al.  Comments on 'A Critical Look' , 1991, IEEE Softw..

[9]  Terence P. Rout,et al.  SPICE: A Framework for Software Process Assessment , 1995, Softw. Process. Improv. Pract..

[10]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[11]  Margaret Tan,et al.  Impact of Organisational Maturity on Software Quality , 1994, Software Quality and Productivity.

[12]  M. J. Allen.  Introduction to Measurement Theory , 1979 .

[13]  Khaled El Emam,et al.  Spice: The Theory and Practice of Software Process Improvement and Capability Determination , 1997 .

[14]  Sasa M. Dekleva,et al.  Measuring Software Engineering Maturity: A Rasch Calibration , 1993, ICIS.

[15]  Watts S. Humphrey,et al.  Conducting SEI-Assisted Software Process Assessments , 1989 .

[16]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[17]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[18]  Rick Barbour,et al.  Software Capability Evaluation, Version 3.0. Implementation Guide for Supplier Selection. , 1996 .

[19]  Khaled El Emam,et al.  An Empirical Evaluation of the Prospective International SPICE Standard , 1996, Softw. Process. Improv. Pract..

[20]  Dennis R. Goldenson,et al.  SPICE: an empiricist's perspective , 1995, Proceedings of Software Engineering Standards Symposium.

[21]  Watts S. Humphrey,et al.  A comparison of US and Japanese software process maturity , 1991, [1991 Proceedings] 13th International Conference on Software Engineering.

[22]  Dennis R. Goldenson,et al.  Interrater agreement in SPICE-based assessments: some preliminary results , 1996, Proceedings of Software Process 1996.

[23]  Khaled El Emam,et al.  Does Organizational Maturity Improve Quality? , 1996, IEEE Softw..

[24]  Donna K. Dunaway,et al.  CMM-Based Appraisal for Internal Process Improvement (CBA IPI): Method Description , 1996 .

[25]  Jane Siegel,et al.  Interim Profile: Development and Trial of a Method to Rapidly Measure Software Engineering Maturity Status , 1994 .

[26]  Lionel C. Briand,et al.  Assessor agreement in rating SPICE processes , 1996, Softw. Process. Improv. Pract..