A quantitative and qualitative analysis of factors affecting software processes

Despite the growing body of research on software process improvement (SPI), there is still a great deal of variability in the success of SPI programmes. In this paper, we explore 26 factors that potentially affect SPI. We also consider the research strategies used to study these factors. We have used a multi-strategy approach for this study: first, by combining qualitative and quantitative analysis within case studies; second, by comparing our case study results with the results of a previously conducted survey study. Seven factors relevant to SPI (i.e. executive support, experienced staff, internal process ownership, metrics, procedures, reviews, and training) were identified by the case studies and the survey study. Two factors (reward schemes and estimating tools) were found, by both the case studies and the survey study, not to be relevant to SPI. Three additional factors (people, problems and change) were identified by the case studies. The frequency with which people, problems and change are discussed by practitioners suggests that these three factors may be pervasive in SPI, in a way that the other factors are not. These factors, however, require further investigation.

[1]  Anita D. Carleton,et al.  Case studies of software-process-improvement measurement , 1994, Computer.

[2]  Richard Messnarz,et al.  Better Software Practice for Business Benefit: Principles and Experiences , 1999 .

[3]  Anselm L. Strauss,et al.  Qualitative Analysis For Social Scientists , 1987 .

[4]  Hajime Enomoto,et al.  Software development process from natural language specification , 1989, ICSE '89.

[5]  William A. Florac,et al.  Statistical Process Control: Analyzing a Space Shuttle Onboard Software Process , 2000, IEEE Softw..

[6]  Nathan Baddoo,et al.  Towards implementing successful software inspections , 2000, Proceedings International Conference on Software Methods and Tools. SMT 2000.

[7]  Mark C. Paulk,et al.  The 1999 Survey of High Maturity Organizations , 2000 .

[8]  Dan Nash,et al.  Raytheon Electronic Systems expe-rience in software process improvement , 1995 .

[9]  Mark C. Paulk,et al.  The Capability Maturity Model: Guidelines for Improving the Software Process , 1994 .

[10]  R. Dion,et al.  Elements of a process-improvement program (software quality) , 1992, IEEE Software.

[11]  S. Raman It is software process, stupid: next millennium software quality key , 2000 .

[12]  Austen Rainer,et al.  Key success factors for implementing software process improvement: a maturity-based analysis , 2002, J. Syst. Softw..

[13]  Kathleen M. Carley Extracting team mental models through textual analysis , 1997 .

[14]  Thomas J. Haley,et al.  Software Process Improvement At Raytheon , 1996, IEEE Softw..

[15]  Kouichi Kishida,et al.  Nontechnological issues in software engineering , 1991, [1991 Proceedings] 13th International Conference on Software Engineering.

[16]  Robert Oshana,et al.  Tailoring Cleanroom for Industrial Use , 1998, IEEE Softw..

[17]  D. Bromley,et al.  Personality description in ordinary language , 1977 .

[18]  O. Holsti Content Analysis for the Social Sciences and Humanities , 1969 .

[19]  Austen Rainer,et al.  An analysis of some core studies of software process improvement , 2001, Softw. Process. Improv. Pract..

[20]  Werner Mellis,et al.  Success factors of organizational change in software process improvement , 1998 .

[21]  Tracy Hall,et al.  Measurement in Software Process Improvement Programmes: An Empirical Study , 2000, IWSM.

[22]  Judith G. Brodman,et al.  What small business and small organizations say about the CMM: experience report , 1994, ICSE '94.

[23]  Victor R. Basili,et al.  The SEL Adapts to Meet Changing Times , 1997 .

[24]  David H. Kitson,et al.  An analysis of SEI software process assessment results: 1987-1991 , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[25]  Peter P. Chen English Sentence Structure and Entity-Relationship Diagrams , 1983, Inf. Sci..

[26]  Bill Pitterman Telcordia Technologies: The Journey to High Maturity , 2000, IEEE Softw..

[27]  Raymond Dion,et al.  Process improvement and the corporate balance sheet , 1993, IEEE Software.

[28]  Brian Fitzgerald,et al.  A Longitudinal Study of Software Process Improvement , 1999, IEEE Softw..

[29]  George Yamamura Process Improvement Satisfies Employees , 1999, IEEE Softw..

[30]  Michael Daskalantonakis,et al.  A Practical View of Software Measurement and Implementation Experiences Within Motorola , 1992, IEEE Trans. Software Eng..

[31]  Nathan Baddoo,et al.  Implementing a people focused SPI programme , 2001 .

[32]  R. Yin Case Study Research: Design and Methods , 1984 .

[33]  David Zubrow,et al.  Moving On Up: Data and Experience Doing CMM-Based Process Improvement , 1995 .

[34]  Michael Diaz,et al.  How Software Process Improvement Helped Motorola , 1997, IEEE Softw..

[35]  C. Billings,et al.  Journey to a Mature Software Process , 1994, IBM Syst. J..

[36]  Aileen B. Sedman,et al.  A longitudinal study , 1987 .

[37]  Watts S. Humphrey,et al.  Software process improvement at Hughes Aircraft , 1991, IEEE Software.

[38]  Tilo Messer,et al.  Siemens process assessment and improvement approaches: experiences and benefits , 1998, Proceedings. The Twenty-Second Annual International Computer Software and Applications Conference (Compsac '98) (Cat. No.98CB 36241).

[39]  Marvin V. Zelkowitz,et al.  Software Process Improvement in the NASA Software Engineering Laboratory , 1994 .

[40]  Girish Seshagiri,et al.  Software Process Improvement Works , 1999 .

[41]  J. G. Brodman,et al.  What small businesses and small organizations say about the CMM , 1994, Proceedings of 16th International Conference on Software Engineering.

[42]  Russ Abbott Program design by informal English descriptions , 1983, CACM.

[43]  Khaled El Emam,et al.  Elements of Software Process Assessment & Improvement , 1999 .

[44]  James D. Herbsleb,et al.  Benefits of CMM-Based Software Process Improvement: Initial Results , 1994 .

[45]  Donatella Sciuto,et al.  Pursuing coherence in software process assessment and improvement , 2001, Softw. Process. Improv. Pract..

[46]  Susan Rosenbaum,et al.  Software improvements in an international company , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[47]  James D. Herbsleb,et al.  After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success. , 1995 .

[48]  R. R. Willis,et al.  Hughes Aircraft's Widespread Deployment of a Continuously Improving Software Process , 1998 .

[49]  Tore Dybå,et al.  An Instrument for Measuring the Key Factors of Success in Software Process Improvement , 2000, Empirical Software Engineering.

[50]  Michael Daskalantonakis,et al.  Achieving higher SEI levels , 1994, IEEE Software.

[51]  Mark M. Misic,et al.  Reading between the lines: an examination of systems analysis and design texts , 2000, J. Syst. Softw..

[52]  Egon Berghout,et al.  From process improvement to people improvement: enabling learning in software development , 2000, Inf. Softw. Technol..

[53]  Edward De Bono,et al.  De Bono's Thinking Course , 1982 .

[54]  Kelley Butler,et al.  Software Process Achievement at Tinker Air Force Base , 2000 .

[55]  Carlo Ghezzi,et al.  Software processes: a retrospective and a path to the future , 1998, Softw. Process. Improv. Pract..

[56]  J. Herbsleb,et al.  A systematic survey of CMM experience and results , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[57]  Susan Rosenbaum,et al.  Schlumberger's Software Improvement Program , 1994, IEEE Trans. Software Eng..

[58]  Austen Rainer,et al.  An Overview Of The Practitioners, Processes and Products Project , 2001 .