Characteristics of multiple-component defects and architectural hotspots: a large system case study

The architecture of a large software system is widely considered important for such reasons as: providing a common goal to the stakeholders in realising the envisaged system; helping to organise the various development teams; and capturing foundational design decisions early in the development. Studies have shown that defects originating in system architectures can consume twice as much correction effort as that for other defects. Clearly, then, scientific studies on architectural defects are important for their improved treatment and prevention. Previous research has focused on the extent of architectural defects in software systems. For this paper, we were motivated to ask the following two complementary questions in a case study: (i) How do multiple-component defects (MCDs)—which are of architectural importance—differ from other types of defects in terms of (a) complexity and (b) persistence across development phases and releases? and (ii) How do highly MCD-concentrated components (the so called, architectural hotspots) differ from other types of components in terms of their (a) interrelationships and (b) persistence across development phases and releases? Results indicate that MCDs are complex to fix and are persistent across phases and releases. In comparison to a non-MCD, a MCD requires over 20 times more changes to fix it and is 6 to 8 times more likely to cross a phase or a release. These findings have implications for defect detection and correction. Results also show that 20% of the subject system’s components contain over 80% of the MCDs and that these components are 2–3 times more likely to persist across multiple system releases than other components in the system. Such MCD-concentrated components constitute architectural “hotspots” which management can focus upon for preventive maintenance and architectural quality improvement. The findings described are from an empirical study of a large legacy software system of size over 20 million lines of code and age over 17 years.

[1]  Michael Eonsuk Shin,et al.  Detection of anomalies in software architecture with connectors , 2006, Sci. Comput. Program..

[2]  Hany H. Ammar,et al.  Software architectures change propagation tool (SACPT) , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[3]  Forrest Shull,et al.  Evolving Defect "Folklore": A Cross-Study Analysis of Software Defect Behavior , 2005, ISPW.

[4]  Andreas Zeller,et al.  Predicting component failures at design time , 2006, ISESE '06.

[5]  Leonard J. Bass,et al.  Summary for leadership and management in software architecture (lMSA 2008) , 2008, ICSE Companion '08.

[6]  Martin Fowler.  Design - Who needs an architect? , 2003, IEEE Software.

[7]  Václav Rajlich,et al.  Removing clones from the code , 1999, J. Softw. Maintenance Res. Pract..

[8]  Henry Muccini,et al.  Software architecture-based regression testing , 2006, J. Syst. Softw..

[9]  Gail C. Murphy,et al.  Predicting source code changes by mining change history , 2004, IEEE Transactions on Software Engineering.

[10]  James H. Cross,et al.  Reverse engineering and design recovery: a taxonomy , 1990, IEEE Software.

[11]  Catherine Stringfellow,et al.  Quantitative Analysis of Development Defects to Guide Testing: A Case Study , 2001, Software Quality Journal.

[12]  David M. Weiss,et al.  Evaluating software development by error analysis: The data from the Architecture Research Facility , 1984, J. Syst. Softw..

[13]  Richard C. Holt,et al.  Software architecture transformations , 2000, Proceedings 2000 International Conference on Software Maintenance.

[14]  Nenad Medvidovic,et al.  Reconciling software requirements and architectures: the CBSP approach , 2001, Proceedings Fifth IEEE International Symposium on Requirements Engineering.

[15]  David Garlan,et al.  Documenting software architectures: views and beyond , 2002, 25th International Conference on Software Engineering, 2003. Proceedings..

[16]  Christof Ebert.  Experiences with criticality predictions in software development , 1997, ESEC '97/FSE-5.

[17]  Uldis Donins,et al.  Reconciling software requirements and architectures within MDA , 2009, Sci. J. Riga Tech. Univ. Ser. Comput. Sci..

[18]  Leonard J. Bass,et al.  Leadership and management in software architecture (LMSA'08): a report on an ICSE workshop , 2008, SOEN.

[19]  Victor R. Basili,et al.  Software errors and complexity: an empirical investigation , 1993 .

[20]  Martin Höst,et al.  The architectural change process , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[21]  Mohammad El-Ramly,et al.  Experience in teaching a software reengineering course , 2006, ICSE.

[22]  Felix Bachmann,et al.  Preliminary Design of ArchE: A Software Architecture Design Assistant , 2003 .

[23]  Claes Wohlin,et al.  Deriving fault architectures from defect history , 2000 .

[24]  Victor R. Basili,et al.  Evaluating Software Development by Analysis of Changes: Some Data from the Software Engineering Laboratory , 1985, IEEE Transactions on Software Engineering.

[25]  Andreas Zeller,et al.  Mining Version Histories to Guide Software Changes , 2004 .

[26]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..

[27]  Eila Niemelä,et al.  A Survey on Software Architecture Analysis Methods , 2002, IEEE Trans. Software Eng..

[28]  Catherine Stringfellow,et al.  Comparison of software architecture reverse engineering methods , 2006, Inf. Softw. Technol..

[29]  Hitoshi Kume,et al.  A Case History Analysis of Software Error Cause-Effect Relationships , 1991, IEEE Trans. Software Eng..

[30]  Paul Clements,et al.  Software Architecture in Practice: Addison-Wesley , 1998 .

[31]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[32]  Per Runeson,et al.  A Second Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems , 2007, IEEE Transactions on Software Engineering.

[33]  Liam O'Brien,et al.  Architecture Reconstruction Guidelines , 2001 .

[34]  Christof Ebert,et al.  Best Practices in Software Measurement , 2004 .

[35]  Norman E. Fenton,et al.  Quantitative Analysis of Faults and Failures in a Complex Software System , 2000, IEEE Trans. Software Eng..

[36]  Dewayne E. Perry,et al.  EMPIRICAL STUDY OF SOFTWARE INTERFACE FAULTS - AN UPDATE. , 1987 .

[37]  Grady Booch,et al.  The Economics of Architecture-First , 2007, IEEE Software.

[38]  Paola Inverardi,et al.  Architecture-based software testing , 1996, ISAW '96.

[39]  Andriy V. Miranskyy,et al.  Analysis of pervasive multiple-component defects in a large software system , 2009, 2009 IEEE International Conference on Software Maintenance.

[40]  Yong Kim,et al.  The vital few versus the trivial many: examining the Pareto principle for software , 2005, 29th Annual International Computer Software and Applications Conference (COMPSAC'05).

[41]  Salvatore Valenti,et al.  Successful Software Reengineering , 2002 .

[42]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[43]  Grady Booch.  Nine Things You Can Do with Old Software , 2008, IEEE Software.

[44]  Per Runeson,et al.  A Replicated Quantitative Analysis of Fault Distributions in Complex Software Systems , 2007, IEEE Transactions on Software Engineering.

[45]  Edward N. Adams,et al.  Optimizing Preventive Service of Software Products , 1984, IBM J. Res. Dev..

[46]  Richard Fanta,et al.  Removing clones from the code , 1999 .

[47]  A. Twycross.  Research design: qualitative, quantitative and mixed methods approaches Research design: qualitative, quantitative and mixed methods approaches Creswell John W Sage 320 £29 0761924426 0761924426 [Formula: see text]. , 2004, Nurse researcher.

[48]  Carol Withrow,et al.  Prediction and control of ADA software defects , 1990, J. Syst. Softw..

[49]  Tze-Jie Yu,et al.  An Analysis of Several Software Defect Models , 1988, IEEE Trans. Software Eng..

[50]  Weider D. Yu.  A software fault prevention approach in coding and root cause analysis , 1998, Bell Labs Technical Journal.

[51]  Barry Boehm,et al.  Top 10 list [software development] , 2001 .

[52]  Claes Wohlin,et al.  Identification of green, yellow and red legacy components , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[53]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[54]  Elaine J. Weyuker,et al.  The distribution of faults in a large industrial software system , 2002, ISSTA '02.

[55]  H. Fawcett.  Manual of Political Economy , 1995 .

[56]  Jan Bosch,et al.  Software Architecture as a Set of Architectural Design Decisions , 2005, 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05).

[57]  Paul Clements,et al.  Software Architecture: An Executive Overview , 1996 .

[58]  Victor R. Basili,et al.  Software errors and complexity: an empirical investigation0 , 1984, CACM.

[59]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[60]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[61]  Albert Endres,et al.  An analysis of errors and their causes in system programs , 1975, IEEE Transactions on Software Engineering.

[62]  Paola Inverardi,et al.  Formal Specification and Analysis of Software Architectures Using the Chemical Abstract Machine Model , 1995, IEEE Trans. Software Eng..

[63]  Dewayne E. Perry,et al.  A case study in root cause defect analysis , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[64]  Niclas Ohlsson,et al.  Predicting Fault-Prone Software Modules in Telephone Switches , 1996, IEEE Trans. Software Eng..

[65]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.