Software process improvement via ISO 9000? Results of two surveys among European software houses

Presents the results of two surveys among European software houses. One of the goals was to find out whether software companies have actually achieved software process improvements via ISO 9000. The first survey was conducted among 20 German software houses that have received an ISO 9001 certificate. The study focuses on five elements of an ISO 9000 quality system: code reviews and inspections software testing, product and process measurements, measurement of quality costs, and demonstration of quality improvements. Many software houses included in our first survey have not carried out any modifications of the five elements. Thus, it seemed that ISO 9000 had not led to significant improvements. Nevertheless, nearly 100% of the companies would decide in favor of implementing an ISO 9000 quality system once again. We decided to conduct a second study to gain a better understanding of the software process improvements achieved via ISO 9000. We analyzed experience reports and conducted interviews with quality managers from a total of 36 European software houses. It turned out that software houses have indeed achieved process improvements. Unfortunately, most software houses are not able to quantify the benefits of implementing an ISO 9000 quality system. We identified ten key success factors that the respondents of our studies considered to be the most helpful when implementing an ISO 9000 quality system. Astonishingly, only two of the ten factors are explicit requirements of ISO 9001. This shows that it is necessary to implement a more comprehensive approach to achieve substantial software process improvements.

[1]  David N. Card,et al.  Learning from Our Mistakes with Defect Causal Analysis , 1999, IEEE Softw..

[2]  Durward K. Sobek,et al.  Another look at how Toyota integrates product development , 1998 .

[3]  Gerald M. Weinberg,et al.  Egoless Programming (Excerpt from the Psychology of Computer Programming, Silver Anniversary Edition). , 1998 .

[4]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[5]  R. R. Willis,et al.  Hughes Aircraft's Widespread Deployment of a Continuously Improving Software Process , 1998 .

[6]  Robert C. Newbold,et al.  Project Management in the Fast Lane: Applying the Theory of Constraints , 1998 .

[7]  U Dave,et al.  Critical Chain , 1998, J. Oper. Res. Soc..

[8]  Norm Bridge,et al.  Orthogonal Defect Classification Using Defect Data to Improve Software Development , 1998 .

[9]  Andrew Hale,et al.  Against the gods: the remarkable story of risk , 1997 .

[10]  Steve McConnell,et al.  Software Project Survival Guide , 1997 .

[11]  James D. Herbsleb,et al.  Software quality and the Capability Maturity Model , 1997, CACM.

[12]  Robert B. Grady,et al.  Successful Software Process Improvement , 1997 .

[13]  Gerald M. Weinberg,et al.  Quality software management: volume 4: anticipating change , 1997 .

[14]  Barbara A. Kitchenham,et al.  Estimates, Uncertainty, and Risk , 1997, IEEE Softw..

[15]  Audrey J. Dorofee,et al.  Putting Risk Management Into Practice , 1997, IEEE Softw..

[16]  Kim Caputo.  Cmm Implementation Guide: Choreographing Software Process Improvement , 1997 .

[17]  Robert P. Smith,et al.  The historical roots of concurrent engineering fundamentals , 1997 .

[18]  Henk Sol,et al.  Proceedings of the 54th Hawaii International Conference on System Sciences , 1997, HICSS 2015.

[19]  Barry Boehm,et al.  The effects of software process maturity on software development effort , 1997 .

[20]  P. Kidwell,et al.  The mythical man-month: Essays on software engineering , 1996, IEEE Annals of the History of Computing.

[21]  Ann Miller,et al.  Selecting Software Subcontractors , 1996, IEEE Softw..

[22]  Shari Lawrence Pfleeger.  Measuring Reuse: A Cautionary Tale , 1996, IEEE Softw..

[23]  Tom DeMarco,et al.  A defined process for project post mortem review , 1996, IEEE Software.

[24]  Jeffrey Pfeffer,et al.  When it comes to “best practices” — Why do smart organizations occasionally do dumb things? , 1996 .

[25]  Joseph D. Blackburn,et al.  Concurrent software engineering: prospects and pitfalls , 1996 .

[26]  Dewayne E. Perry,et al.  Barriers to Effective Process Architecture – An Experience Report , 1996 .

[27]  Stephen H. Kan,et al.  Metrics and Models in Software Quality Engineering , 1994, SOEN.

[28]  John Micklethwait,et al.  The Witch Doctors: Making Sense of the Management Gurus , 1996 .

[29]  W. Duncan.  A GUIDE TO THE PROJECT MANAGEMENT BODY OF KNOWLEDGE , 1996 .

[30]  The Challenge of "Good Enough" Software , 1996 .

[31]  Thomas O. Jones,et al.  Why Satisfied Customers Defect , 1996 .

[32]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[33]  Michael Schrage,et al.  No More Teams!: Mastering the Dynamics of Creative Collaboration , 1995 .

[34]  J. Bach.  Enough About Process: What We Need are Heroes , 1995, IEEE Softw..

[35]  Ann Langley,et al.  Between 'Paralysis by Analysis' and 'Extinction by Instinct' , 1995 .

[36]  Larry Constantine,et al.  Constantine on Peopleware , 1995 .

[37]  Frederik D. Wiersema,et al.  The Discipline of Market Leaders , 1995 .

[38]  Hossein Saiedian,et al.  SEI Capability Maturity Model's Impact on Contractors , 1995, Computer.

[39]  Robert E. Park.  A Manager's Checklist for Validating Software Cost and Schedule Estimates , 1995 .

[40]  Tom DeMarco.  Why Does Software Cost So Much , 1995 .

[41]  Marvin V. Zelkowitz,et al.  Software Process Improvement in the NASA Software Engineering Laboratory , 1994 .

[42]  Susan Rosenbaum,et al.  Schlumberger's Software Improvement Program , 1994, IEEE Trans. Software Eng..

[43]  George E. Stark,et al.  Using metrics in management decision making , 1994, Computer.

[44]  Anita D. Carleton,et al.  Case studies of software-process-improvement measurement , 1994, Computer.

[45]  Robert B. Grady,et al.  Key lessons in achieving widespread inspection use , 1994, IEEE Software.

[46]  Michael Brassard,et al.  The Memory Jogger II , 1994 .

[47]  Gerald M. Weinberg,et al.  Quality Software Management, Vol. 3: Congruent Action , 1994 .

[48]  Joseph O. Voelkel.  Understanding Variation: The Key to Managing Chaos , 1994 .

[49]  C. Billings,et al.  Journey to a Mature Software Process , 1994, IBM Syst. J..

[50]  Capers Jones,et al.  Assessment and control of software risks , 1994, Yourdon Press Computing Series.

[51]  Inderpal S. Bhandari,et al.  A Case Study of Software Process Improvement During Development , 1993, IEEE Trans. Software Eng..

[52]  John C. Knight,et al.  An improved inspection technique , 1993, CACM.

[53]  Edward F. Weller,et al.  Lessons from three years of inspection data (software development) , 1993, IEEE Software.

[54]  Wei-Tek Tsai,et al.  Distributed, collaborative software inspection , 1993, IEEE Software.

[55]  D. Garvin.  Building a learning organization. , 1993, Harvard business review.

[56]  D Niven,et al.  When times get tough, what happens to TQM? Case study. , 1993, Harvard business review.

[57]  Mark C. Paulk,et al.  Capability Maturity Model for Software, Version 1.1 , 1993 .

[58]  Suzanne Garcia,et al.  Key Practices of the Capability Maturity Model, Version 1.1 , 1993 .

[59]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..

[60]  Bill Curtis,et al.  Process modeling , 1992, CACM.

[61]  Albert L. Lederer,et al.  Nine management guidelines for better cost estimating , 1992, CACM.

[62]  R. Schaffer,et al.  Successful change programs begin with results. , 1992, Harvard business review.

[63]  Watts S. Humphrey,et al.  CASE planning and the software process , 1991, J. Syst. Integr..

[64]  Mark C. Paulk,et al.  Key Practices of the Capability Maturity Model , 1991 .

[65]  Alan M. Davis,et al.  Impacts of life cycle models on software configuration management , 1991, CACM.

[66]  Watts S. Humphrey,et al.  Comments on 'A Critical Look' , 1991, IEEE Softw..

[67]  Terry Bollinger,et al.  A critical look at software capability evaluations , 1991, IEEE Software.

[68]  Barbara Farbey,et al.  Exploring Requirements: Quality Before Design , 1991 .

[69]  M. Beer,et al.  Why change programs don't produce change. , 1990, Harvard business review.

[70]  Wiliam Wiggenhorn,et al.  Motorola U: when training becomes an education , 1990 .

[71]  David I. Cleland,et al.  Project Management: Strategic Design and Implementation , 1990 .

[72]  Jesus R. Tirso,et al.  Implementing the Defect Prevention Process in the MVS Interactive Programming Organization , 1990, IBM Syst. J..

[73]  Robert G. Mays,et al.  Experiences with Defect Prevention , 1990, IBM Syst. J..

[74]  A. Frank Ackerman,et al.  Software inspections: an effective verification process , 1989, IEEE Software.

[75]  John D. Musa,et al.  Quantifying Software Validation: When to Stop Testing? , 1989, IEEE Softw..

[76]  Dick Bowman,et al.  Principles of software engineering management , 1989, APLQ.

[77]  Watts S. Humphrey,et al.  Conducting SEI-Assisted Software Process Assessments , 1989 .

[78]  F. J. Buckley.  The roles of a SQA person , 1987, SOEN.

[79]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[80]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[81]  Bill Curtis,et al.  On building software process models under the lamppost , 1987, ICSE '87.

[82]  Michael E. Fagan.  Advances in software inspections , 1986, IEEE Transactions on Software Engineering.

[83]  Stuart E. Madnick,et al.  Special Feature: Impact of Schedule Estimation on Software Project Behavior , 1986, IEEE Software.

[84]  Peter F. Drucker,et al.  Innovation and Entrepreneurship: Practices and Principles , 1986 .

[85]  Paul Rook,et al.  Controlling software projects , 1986, Softw. Eng. J..

[86]  David Lorge Parnas,et al.  Active design reviews: principles and practices , 1985, ICSE '85.

[87]  Carole L. Jones.  A Process-Integrated Approach to Defect Prevention , 1985, IBM Syst. J..

[88]  Edward H. Bersoff,et al.  Elements of Software Configuration Management , 1984, IEEE Transactions on Software Engineering.

[89]  Robert M. Poston,et al.  Software Quality Assurance , 1984, IEEE Transactions on Software Engineering.

[90]  A. Marty.  Getting to YES. Negotiating Agreement Without Giving In , 1983 .

[91]  J. Voelkel.  Guide to Quality Control , 1982 .

[92]  C. Handy.  Gods of Management: The Changing Work of Organizations , 1979 .

[93]  E. Bersoff,et al.  Software Configuration Management , 1978, SIGMETRICS Perform. Evaluation Rev..

[94]  John W. Meyer,et al.  Institutionalized Organizations: Formal Structure as Myth and Ceremony , 1977, American Journal of Sociology.

[95]  J W Borthwick,et al.  The human side of management. , 1976, The Canadian Journal of Hospital Pharmacy.

[96]  Alan F. Blackwell,et al.  Programming , 1973, CSC '73.

[97]  Robert Townsend.  Up the Organization: How to Stop the Corporation from Stifling People and Strangling Profits , 1970 .

[98]  W. Powell,et al.  The iron cage revisited institutional isomorphism and collective rationality in organizational fields , 1983 .