An analysis of some core studies of software process improvement

In this paper we analyse 39 publications that have reported issues relating to software process improvement. Together, the 39 publications refer to 14 sites in 11 organizations. The 11 organizations include all five of the organizations that have been awarded the IEEE Computer Society Award for Process Achievement. Twelve of the 14 sites have been assessed at CMM level 3 or higher. Two main issues, and a number of more specific issues, emerge from our analysis. The two main issues are organizational stability and process expertise. We further distinguish between inter- and intra-organizational stability. Several other specific issues are identified and discussed. We direct attention at how the specific issues relate to each other, how the two main issues relate to each other, and how all these issues relate to software process improvement. We speculate that organizational stability provides a stable environment within which to perform the process and to develop expertise. In addition to the 39 publications, the paper provides cross-references to survey studies and other case studies on process improvement, and studies into process understanding. Copyright © 2001 John Wiley & Sons, Ltd.

[1]  Wolfgang Hesse,et al.  Results from the IPAS project: Influences of methods and tools, quality requirements and project management on the work situation of software developers , 1992 .

[2]  Claude Y. Laporte,et al.  Addressing the people issues of process improvement activities at Oerlikon Aerospace , 1998, Softw. Process. Improv. Pract..

[3]  Thomas J. Haley,et al.  Software Process Improvement At Raytheon , 1996, IEEE Softw..

[4]  G. Kenni,et al.  The evolution of quality processes at Tata Consultancy Services , 2000 .

[5]  Michiel van Genuchten,et al.  Why is Software Late? An Empirical Study of Reasons For Delay in Software Development , 1991, IEEE Trans. Software Eng..

[6]  Judith G. Brodman,et al.  Return on Investment (ROI) from Software Process Improvement as Measured by US Industry , 1995, Softw. Process. Improv. Pract..

[7]  Chris W. Clegg,et al.  The dynamics of work organization, knowledge and technology during software development , 1997, Int. J. Hum. Comput. Stud..

[8]  David N. Wilson,et al.  A framework for evaluation and prediction of software process improvement success , 2001, J. Syst. Softw..

[9]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[10]  William A. Florac,et al.  Statistical Process Control: Analyzing a Space Shuttle Onboard Software Process , 2000, IEEE Softw..

[11]  Kelley Butler,et al.  Software Process Achievement at Tinker Air Force Base , 2000 .

[12]  Judith S. Olson,et al.  Small Group Design Meetings: An Analysis of Collaboration , 1992, Hum. Comput. Interact..

[13]  David Zubrow,et al.  Moving On Up: Data and Experience Doing CMM-Based Process Improvement , 1995 .

[14]  Susan Rosenbaum,et al.  Software improvements in an international company , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[15]  James D. Herbsleb,et al.  After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success. , 1995 .

[16]  Alexander L. Wolf,et al.  Cost-Effective Analysis of In-Place Software Processes , 1998, IEEE Trans. Software Eng..

[17]  Dewayne E. Perry,et al.  Prototyping a process monitoring experiment , 1994, Proceedings of 1993 15th International Conference on Software Engineering.

[18]  Deependra Moitra,et al.  Managing change for software process improvement initiatives: a practical experience-based approach , 1998, Softw. Process. Improv. Pract..

[19]  Federico Vazquez,et al.  Automatic system file generator , 1993, SOEN.

[20]  Anita D. Carleton,et al.  Case studies of software-process-improvement measurement , 1994, Computer.

[21]  Victor R. Basili,et al.  The SEL Adapts to Meet Changing Times , 1997 .

[22]  R. R. Willis,et al.  Hughes Aircraft's Widespread Deployment of a Continuously Improving Software Process , 1998 .

[23]  Werner Mellis,et al.  Success factors of organizational change in software process improvement , 1998 .

[24]  Marilyn W. Bush.  Process assessments in NASA , 1991, [1991 Proceedings] 13th International Conference on Software Engineering.

[25]  Gargi Keeni,et al.  The Evolution of Quality Processes at Tata Consultancy Services , 2000, IEEE Softw..

[26]  R. Dion,et al.  Elements of a process-improvement program (software quality) , 1992, IEEE Software.

[27]  Mark C. Paulk,et al.  Capability Maturity Model for Software , 2001 .

[28]  S. Raman.  It is software process, stupid: next millennium software quality key , 2000 .

[29]  J. Bach.  Enough About Process: What We Need are Heroes , 1995, IEEE Softw..

[30]  David H. Kitson,et al.  An analysis of SEI software process assessment results: 1987-1991 , 1993, Proceedings of 1993 15th International Conference on Software Engineering.

[31]  Dewayne E. Perry,et al.  People, organizations, and process improvement , 1994, IEEE Software.

[32]  Harvey Wohlwend,et al.  A Case History of the Space Shuttle Onboard Systems Project Technology Transfer , 1994 .

[33]  Bill Curtis,et al.  The global pursuit of process maturity , 2000 .

[34]  James Coplien,et al.  A role-based empirical process modeling environment , 1993, [1993] Proceedings of the Second International Conference on the Software Process-Continuous Software Process Improvement.

[35]  Robert Oshana,et al.  Tailoring Cleanroom for Industrial Use , 1998, IEEE Softw..

[36]  F. Mcgarry.  Overview of the Software Engineering Laboratory , 1979 .

[37]  Dewayne E. Perry,et al.  Understanding and Improving Time Usage in Software Development , 1995 .

[38]  Michael Diaz,et al.  How Software Process Improvement Helped Motorola , 1997, IEEE Softw..

[39]  Mark C. Paulk,et al.  Capability Maturity Model , 1991 .

[40]  Wolfgang Hesse.  Theory and Practice of the Software Process: A Field Study and Its Implications for Project Management , 1996, EWSPT.

[41]  Susan Rosenbaum,et al.  Schlumberger's Software Improvement Program , 1994, IEEE Trans. Software Eng..

[42]  Mark C. Paulk,et al.  The 1999 Survey of High Maturity Organizations , 2000 .

[43]  Dan Nash,et al.  Raytheon Electronic Systems expe-rience in software process improvement , 1995 .

[44]  Alexander L. Wolf,et al.  Process discovery and validation through event-data analysis , 1996 .

[45]  Harvey P. Siy,et al.  An experiment to assess the cost-benefits of code inspections in large scale software development , 1995, SIGSOFT '95.

[46]  Nathan Baddoo,et al.  Towards implementing successful software inspections , 2000, Proceedings International Conference on Software Methods and Tools. SMT 2000.

[47]  Khaled El Emam,et al.  Costs and Benefits of Software Process Improvement , 1997 .

[48]  Raymond Dion,et al.  Process improvement and the corporate balance sheet , 1993, IEEE Software.

[49]  Joyce J. Elam,et al.  Inside a software design team: knowledge acquisition, sharing, and integration , 1993, CACM.

[50]  David S. Rosenblum,et al.  A study in software process data capture and analysis , 1993, [1993] Proceedings of the Second International Conference on the Software Process-Continuous Software Process Improvement.

[51]  Brian Fitzgerald,et al.  A Longitudinal Study of Software Process Improvement , 1999, IEEE Softw..

[52]  Egon Berghout,et al.  Interrupts: Just a Minute Never Is , 1998, IEEE Softw..

[53]  Bill Pitterman.  Telcordia Technologies: The Journey to High Maturity , 2000, IEEE Softw..

[54]  Watts S. Humphrey,et al.  Software process improvement at Hughes Aircraft , 1991, IEEE Software.

[55]  Tilo Messer,et al.  Siemens process assessment and improvement approaches: experiences and benefits , 1998, Proceedings. The Twenty-Second Annual International Computer Software and Applications Conference (Compsac '98) (Cat. No.98CB 36241).

[56]  Sasa M. Dekleva,et al.  Measuring Software Engineering Evolution: A Rasch Calibration , 1997, Inf. Syst. Res..

[57]  Marvin V. Zelkowitz,et al.  Software Process Improvement in the NASA Software Engineering Laboratory , 1994 .

[58]  Girish Seshagiri,et al.  Software Process Improvement Works , 1999 .

[59]  L. G. Votta,et al.  Organizational congestion in large-scale software development , 1994, Proceedings of the Third International Conference on the Software Process. Applying the Software Process.

[60]  George Yamamura.  Process Improvement Satisfies Employees , 1999, IEEE Softw..

[61]  Michael Daskalantonakis,et al.  A Practical View of Software Measurement and Implementation Experiences Within Motorola , 1992, IEEE Trans. Software Eng..

[62]  Judith G. Brodman,et al.  Realities and Rewards of Software Process Improvement , 1996, IEEE Softw..

[63]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[64]  Michael Daskalantonakis,et al.  Achieving higher SEI levels , 1994, IEEE Software.

[65]  Allan Baktoft Jakobsen.  Bottom-up Process Improvement Tricks , 1998, IEEE Softw..

[66]  Karen D. Prenger,et al.  Costs and Benefits of Software Process Improvement , 1997 .

[67]  Mark C. Paulk,et al.  Assessing a Level 5 Organization , 1999 .

[68]  James D. Herbsleb,et al.  Benefits of CMM-Based Software Process Improvement: Initial Results , 1994 .

[69]  C. Billings,et al.  Journey to a Mature Software Process , 1994, IBM Syst. J..

[70]  B. Curtis.  Three Problems Overcome With Behavioral Models Of The Software Development Process , 1989, 11th International Conference on Software Engineering.

[71]  K. Dakin.  Keys to Engineering a Workable Contract , 1996, IEEE Softw..