An Instrument for Measuring the Key Factors of Success in Software Process Improvement

Understandinghow to implement SPI successfully is arguably the most challengingissue facing the SPI field today. The SPI literature containsmany case studies of successful companies and descriptions oftheir SPI programs. However, there has been no systematic attemptto synthesize and organize the prescriptions offered. The researchefforts to date are limited and inconclusive and without adequatetheoretical and psychometric justification.This paper provides a synthesis of prescriptions for successfulquality management and process improvement found from an extensivereview of the quality management, organizational learning, andsoftware process improvement literature. The literature reviewwas confirmed by empirical studies among both researchers andpractitioners. The main result is an instrument for measuringthe key factors of success in SPI based on data collected from120 software organizations. The measures were found to have satisfactorypsychometric properties. Hence, managers can use the instrumentto guide SPI activities in their respective organizations andresearchers can use it to build models to relate the facilitatingfactors to both learning processes and SPI outcomes.

[1]  田口 玄一,et al.  Introduction to quality engineering : designing quality into products and processes , 1986 .

[2]  Jayant V. Saraph,et al.  An Instrument for Measuring the Critical Factors of Quality Management , 1989 .

[3]  Khaled El Emam,et al.  Empirical Studies of Software Process Assessment Methods * † , 1999 .

[4]  John H. Sheesley,et al.  Quality Engineering in Production Systems , 1988 .

[5]  Tony Buzan,et al.  Creating a Thinking Organization: Groundrules for Success , 1999 .

[6]  W. Velicer,et al.  Comparison of five rules for determining the number of components to retain. , 1986 .

[7]  W. French,et al.  Organization development : behavioral science interventions for organization improvement , 1999 .

[8]  N. Tallent.  Psychological testing. , 1960, The American journal of nursing.

[9]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[10]  Timo Saarinen,et al.  An expanded instrument for evaluating information system success , 1996, Inf. Manag..

[11]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[12]  Joseph G. Voelkel,et al.  Guide to Quality Control , 1982 .

[13]  Victor R. Basili,et al.  The role of experimentation in software engineering: past, current, and future , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[14]  野中 郁次郎,et al.  The Knowledge-Creating Company: How , 1995 .

[15]  H. Kaiser.  The Application of Electronic Computers to Factor Analysis , 1960 .

[16]  W. Edwards Deming,et al.  Out of the Crisis , 1982 .

[17]  Khaled El Emam,et al.  The Internal Consistency of the ISO/IEC 15504 Software Process Capability Scale , 1998, IEEE METRICS.

[18]  Peter M. Senge,et al.  The Dance of Change: The Challenges to Sustaining Momentum in Learning Organizations , 1999 .

[19]  Arlene Fink,et al.  How to Conduct Surveys: A Step-by-Step Guide. Sixth Edition. , 1985 .

[20]  B. Price.  A First Course in Factor Analysis , 1993 .

[21]  Tore Dybå.  Improvisation in Small Software Organizations , 2000, IEEE Software.

[22]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[23]  P. Lawrence,et al.  Organization and Environment: Managing Differentiation and Integration , 1967 .

[24]  Massimo Marraffa,et al.  Organizational learning II: Theory, method and practice , 1998 .

[25]  Richard Messnarz,et al.  Better Software Practice for Business Benefit: Principles and Experiences , 1999 .

[26]  Edward G. Schilling,et al.  Juran's Quality Handbook , 1998 .

[27]  James M. Matarazzo,et al.  The knowing organization: How organizations use information to construct meaning, create knowledge, and make decisions: by Chan Wei Choo. New York: Oxford University Press, 1998. 298p. $60.00 (hardcover). ISBN 0-19-511011-0. $25.95 (softcover). ISBN 0-19-511012-9 , 1998 .

[28]  James G. March,et al.  The Pursuit of Organizational Intelligence: Decisions and Learning in Organizations , 1999 .

[29]  H. Blalock,et al.  Theory Construction: From Verbal to Mathematical Formulations. , 1970 .

[30]  R. Gonzalez.  Applied Multivariate Statistics for the Social Sciences , 2003 .

[31]  David W. Zeitler.  Introduction to Quality Control , 1994 .

[32]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[33]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[34]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[35]  Donald A. Schön,et al.  Organizational Learning: A Theory Of Action Perspective , 1978 .

[36]  Eric R. Ziegel,et al.  Total Quality Control , 1992 .

[37]  E. Nevis,et al.  Understanding Organizations as Learning Systems , 1995 .

[38]  Watts S. Humphrey,et al.  Managing Technical People: Innovation, Teamwork, and the Software Process , 1996 .

[39]  K. Arnesen,et al.  In search of the customer's quality view , 1997, J. Syst. Softw..

[40]  Sami Zahran.  Software process improvement - practical guidelines for business success , 1998 .

[41]  M. R. Novick,et al.  Coefficient alpha and the reliability of composite measurements. , 1967, Psychometrika.

[42]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[43]  Harry C. Triandis,et al.  The Human Organization: Its Management and Value. , 1967 .

[44]  Schaffer Rh,et al.  Successful change programs begin with results. , 1992 .

[45]  David P. Baron,et al.  The Nonmarket Strategy System , 1995 .

[46]  Stefan Biffl,et al.  Directions and Methodologies for Empirical Software Engineering Research , 2004, Empirical Software Engineering.

[47]  A. V. D. Ven,et al.  Measuring And Assessing Organizations , 1980 .

[48]  E. Rogers.  Diffusion of Innovations , 1962 .

[49]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[50]  L. Kanuk,et al.  Mail Surveys and Response Rates: A Literature Review , 1975 .

[51]  Donald A. Schön,et al.  Organizational Learning II: Theory, Method, and Practice , 1995 .

[52]  J. March.  The Pursuit of Organizational Intelligence , 1999 .

[53]  Khaled El Emam,et al.  An instrument for measuring the success of the requirements engineering process in information systems development , 2004, Empirical Software Engineering.

[54]  Philip B. Crosby,et al.  Quality Is Still Free: Making Quality Certain in Uncertain Times , 1995 .

[55]  L. J. Porter,et al.  Identification of the Critical Factors of TQM , 1996 .

[56]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[57]  A. Darker,et al.  TOTAL QUALITY CONTROL , 1987 .

[58]  Varun Grover,et al.  Profiling successful reengineering projects , 1998, CACM.

[59]  Gustavo Stubrich.  The Fifth Discipline: The Art and Practice of the Learning Organization , 1993 .

[60]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[61]  Victor R. Basili,et al.  Paradigms for experimentation and empirical studies in software engineering , 1991 .

[62]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability , 2000, IEEE Trans. Software Eng..

[63]  Yehuda Baruch,et al.  Response Rate in Academic Studies — A Comparative Analysis , 1999 .

[64]  Victor R. Basili,et al.  Improve Software Quality by Reusing Knowledge and Experience , 1995 .

[65]  Karlheinz Kautz,et al.  Quality assurance and software process improvement in Norway , 1997 .

[66]  Joseph Moses Juran.  Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services , 1992 .

[67]  M. Appelbaum,et al.  Psychometric methods. , 1989, Annual review of psychology.

[68]  Duane Davis,et al.  Business research for decision making , 1985 .

[69]  Georg Herzwurm,et al.  Software process improvement via ISO 9000? Results of two surveys among European software houses , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[70]  J. Stevens,et al.  Applied Multivariate Statistics for the Social Sciences , 1993 .

[71]  T. C. Powell.  Total Quality Management as Competitive Advantage , 1995 .

[72]  Norman Cliff,et al.  The eigenvalues-greater-than-one rule and the reliability of components. , 1988 .

[73]  I. Nonaka.  A Dynamic Theory of Organizational Knowledge Creation , 1994 .

[74]  Thomas C. Powell,et al.  Total Quality Management as Competitive Advantage , 1995 .

[75]  H. Russell Bernard,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2000 .

[76]  R. Schaffer,et al.  Successful change programs begin with results. , 1992, Harvard business review.

[77]  Bob Smith,et al.  The Internal Consistencies of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension , 1998, Empirical Software Engineering.

[78]  E. Rogers.  Diffusion of Innovations, Fourth Edition , 1982 .

[79]  Sanjay L. Ahire,et al.  Development and Validation of TQM Implementation Constructs , 1996 .

[80]  R. Lissitz,et al.  Effect of the number of scale points on reliability: A Monte Carlo approach. , 1975 .

[81]  James D. Herbsleb,et al.  After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success. , 1995 .

[82]  Tridas Mukhopadhyay,et al.  Software Processes and Project Performance , 1995, J. Manag. Inf. Syst..

[83]  P. Crosby.  Quality Without Tears , 1984 .

[84]  W. Neuman,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2002 .

[85]  W. Deming.  Quality, productivity, and competitive position , 1982 .

[86]  Werner Mellis,et al.  Success factors of organizational change in software process improvement , 1998 .

[87]  H. Kaiser.  A second generation little jiffy , 1970 .

[88]  E. Aspinwall,et al.  Critical success factors for total quality management implementation in small and medium enterprises , 1999 .

[89]  Philip B. Crosby,et al.  Quality Is Free: The Art of Making Quality Certain , 1979 .

[90]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[91]  P. Senge.  The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization , 2014 .

[92]  D. Garvin.  Building a learning organization. , 1993, Harvard business review.

[93]  Watts S. Humphrey,et al.  Managing the software process , 1989, The SEI series in software engineering.

[94]  K. El Emam.  The internal consistency of the ISO/IEC 15504 software process capability scale , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[95]  Khaled El Emam,et al.  Success or Failure ? Modeling the Likelihood of Software Process Improvement , 1998 .

[96]  Paul E. Spector.  Summated rating scale construction , 1991 .