Understanding software maintenance and evolution by analyzing individual changes: a literature review

Understanding, managing and reducing costs and risks inherent in change are key challenges of software maintenance and evolution, addressed in empirical studies with many different research approaches. Change-based studies analyze data that describes the individual changes made to software systems. This approach can be effective in order to discover cost and risk factors that are hidden at more aggregated levels. However, it is not trivial to derive appropriate measures of individual changes for specific measurement goals. The purpose of this review is to improve change-based studies by (1) summarizing how attributes of changes have been measured to reach specific study goals and (2) describing current achievements and challenges, leading to a guide for future change-based studies. Thirty-four papers conformed to the inclusion criteria. Forty-three attributes of changes were identified, and classified according to a conceptual model developed for the purpose of this classification. The goal of each study was to either characterize the evolution process, to assess causal factors of cost and risk, or to predict costs and risks. Effective accumulation of knowledge across change-based studies requires precise definitions of attributes and measures of change. We recommend that new change-based studies base such definitions on the proposed conceptual model. Copyright © 2009 John Wiley & Sons, Ltd. Change-based studies use the individual maintenance task as the primary unit of analysis. This literature review intends to improve change-based studies by summarizing how attributes of changes have been measured to reach specific study goals, and by describing current achievements and challenges, leading to a guide for future studies of this type. Copyright © 2009 John Wiley & Sons, Ltd.

[1]  John G. Keating,et al.  A case study on the adaptive maintenance of an Internet application , 2003, J. Softw. Maintenance Res. Pract..

[2]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[3]  Bin Xu,et al.  Maximizing customer satisfaction in maintenance of software product family , 2005, Canadian Conference on Electrical and Computer Engineering, 2005..

[4]  Zafar I. Malik,et al.  An approach for selective state machine based regression testing , 2007, A-MOST '07.

[5]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[6]  Debra J. Richardson,et al.  Towards leveraging model transformation to support model-based testing , 2007, ASE '07.

[7]  Susan Wiedenbeck,et al.  Empirical studies of software engineering , 2004, Int. J. Hum. Comput. Stud..

[8]  Meir M. Lehman,et al.  A Model of Large Program Development , 1976, IBM Syst. J..

[9]  E. Burton Swanson,et al.  The dimensions of maintenance , 1976, ICSE '76.

[10]  William M. Evanco.  Prediction models for software fault correction effort , 2001, Proceedings Fifth European Conference on Software Maintenance and Reengineering.

[11]  Audris Mockus,et al.  Identifying Productivity Drivers by Modeling Work Units Using Partial Data , 2001, Technometrics.

[12]  Bogdan Korel,et al.  Requirement-based automated black-box test generation , 2001, 25th Annual International Computer Software and Applications Conference. COMPSAC 2001.

[13]  Bogdan Korel,et al.  Model based regression test reduction using dependence analysis , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[14]  Françoise Détienne,et al.  Software Design — Cognitive Aspects , 2001, Practitioner Series.

[15]  Rachida Dssouli,et al.  Automatic executable test case generation for extended finite state machine protocols , 1997 .

[16]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[17]  Bin Xu.  Managing customer satisfaction in maintenance of software product family via ID3 , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[18]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[19]  Tore Dybå,et al.  Building Theories in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[20]  Chris F. Kemerer,et al.  Software complexity and software maintenance: A survey of empirical research , 1995, Ann. Softw. Eng..

[21]  Avishai Wool,et al.  Key management for restricted multicast using broadcast encryption , 2000, TNET.

[22]  Bo Yang,et al.  A test sequence selection method for protocol testing , 1991, IEEE Trans. Commun..

[23]  Audris Mockus,et al.  Inferring change effort from configuration management databases , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[24]  David Notkin,et al.  Checking inside the black box: regression testing by comparing value spectra , 2005, IEEE Transactions on Software Engineering.

[25]  Audris Mockus,et al.  Missing Data in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[26]  H. Dieter Rombach,et al.  Toward full life cycle control: Adding maintenance measurement to the SEL , 1992, J. Syst. Softw..

[27]  Mario Piattini,et al.  An Ontology For The Management Of Software Maintenance Projects , 2004, Int. J. Softw. Eng. Knowl. Eng..

[28]  Dewayne E. Perry,et al.  Toward understanding the rhetoric of small source code changes , 2005, IEEE Transactions on Software Engineering.

[29]  George E. Stark,et al.  Measurements for managing software maintenance , 1996, 1996 Proceedings of International Conference on Software Maintenance.

[30]  Sebastian G. Elbaum,et al.  Code churn: a measure for estimating the impact of code change , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[31]  Sandip C. Patel,et al.  A metrics-based software maintenance effort model , 2004, Eighth European Conference on Software Maintenance and Reengineering, 2004. CSMR 2004. Proceedings..

[32]  Alain Abran,et al.  Measuring the Size of Small Functional Enhancements to Software , 1997 .

[33]  O. Dieste,et al.  Developing Search Strategies for Detecting Relevant Experiments for Systematic Reviews , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[34]  Marek Reformat,et al.  Analysis of software maintenance data using multi-technique approach , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[35]  Hans van Vliet,et al.  Predicting maintenance effort with function points , 1997, 1997 Proceedings International Conference on Software Maintenance.

[36]  Joseph Robert Horgan,et al.  Effect of Test Set Minimization on Fault Detection Effectiveness , 1995, 1995 17th International Conference on Software Engineering.

[37]  Audris Mockus,et al.  Predicting risk of software changes , 2000, Bell Labs Technical Journal.

[38]  Norman F. Schneidewind.  Investigation of the risk to software reliability and maintainability of requirements changes , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[39]  Mary Lou Soffa,et al.  A methodology for controlling the size of a test suite , 1993, TSEM.

[40]  David Chenho Kung,et al.  Class Firewall, Test Order, and Regression Testing of Object-Oriented Programs , 1995, J. Object Oriented Program..

[41]  Victor R. Basili,et al.  Evaluating Software Development by Analysis of Changes: Some Data from the Software Engineering Laboratory , 1985, IEEE Transactions on Software Engineering.

[42]  P. A. Brown,et al.  The application of module regression testing at TRIUMF , 1990 .

[43]  M. S. Krishnan,et al.  An Empirical Analysis of Productivity and Quality in Software Products , 2000 .

[44]  Norman E. Fenton,et al.  Measurement : A Necessary Scientific Basis , 2004 .

[45]  David Gefen,et al.  The non-homogeneous maintenance periods: a case study of software modifications , 1996, 1996 Proceedings of International Conference on Software Maintenance.

[46]  Victor R. Basili,et al.  Software errors and complexity: an empirical investigation0 , 1984, CACM.

[47]  Rajiv D. Banker,et al.  Software complexity and maintenance costs , 1993, CACM.

[48]  D. A. Parker,et al.  The dimensions of healthy maintenance , 1982, ICSE '82.

[49]  Mikael Lindvall.  Monitoring and Measuring the Change-Prediction Process at Different Granularity Levels , 1998 .

[50]  Bo Xie.  Requirement based regression test suite reduction using dependence analysis , 2006 .

[51]  E. Burch,et al.  Modeling software maintenance requests: a case study , 1997, 1997 Proceedings International Conference on Software Maintenance.

[52]  Reidar Conradi,et al.  An empirical study of software change: origin, acceptance rate, and functionality vs. quality attributes , 2004, Proceedings. 2004 International Symposium on Empirical Software Engineering, 2004. ISESE '04..

[53]  Harvey P. Siy,et al.  Parallel changes in large scale software development: an observational case study , 1998, TSEM.

[54]  C. Anantaram,et al.  An influence model for factors in outsourced software maintenance , 2006, J. Softw. Maintenance Res. Pract..

[55]  Chris F. Kemerer,et al.  An Empirical Approach to Studying Software Evolution , 1999, IEEE Trans. Software Eng..

[56]  Daniel M. Germán,et al.  An empirical study of fine-grained software modifications , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[57]  Elaine J. Weyuker.  The Complexity of Data Flow Criteria for Test Data Selection , 1984, Inf. Process. Lett..

[58]  Mario Piattini,et al.  Using code metrics to predict maintenance of legacy programs: a case study , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[59]  Audris Mockus,et al.  An Empirical Study of Speed and Communication in Globally Distributed Software Development , 2003, IEEE Trans. Software Eng..

[60]  Andrian Marcus,et al.  A case study on the long-term effects of software redocumentation , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[61]  B. Glaser.  The Constant Comparative Method of Qualitative Analysis , 1965 .

[62]  Lionel C. Briand,et al.  Automating impact analysis and regression test selection based on UML designs , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[63]  Dewayne E. Perry,et al.  Classification and evaluation of defects in a project retrospective , 2002, J. Syst. Softw..

[64]  Insup Lee,et al.  Data flow testing as model checking , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[65]  Bennet P. Lientz,et al.  Issues in Software Maintenance , 1983, CSUR.

[66]  Dewayne E. Perry,et al.  Metrics and laws of software evolution-the nineties view , 1997, Proceedings Fourth International Software Metrics Symposium.

[67]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[68]  Lionel C. Briand,et al.  Predicting fault-prone components in a java legacy system , 2006, ISESE '06.

[69]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[70]  Gregg Rothermel,et al.  Regression test selection for C++ software , 2000 .

[71]  Lionel C. Briand,et al.  Empirical Studies of Quality Models in Object-Oriented Systems , 2002, Adv. Comput..

[72]  Rajiv Gupta,et al.  An approach to regression testing using slicing , 1992, Proceedings Conference on Software Maintenance 1992.

[73]  Samir Khuller,et al.  Greedy strikes back: improved facility location algorithms , 1998, SODA '98.

[74]  Victor R. Basili,et al.  A classification procedure for the effective management of changes during the maintenance process , 1992, Proceedings Conference on Software Maintenance 1992.

[75]  Anneliese Amschler Andrews,et al.  Program Comprehension During Software Maintenance and Evolution , 1995, Computer.

[76]  Victor R. Basili,et al.  Understanding and predicting the process of software maintenance releases , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[77]  Alain Abran,et al.  Measurement of the maintenance process from a demand-based perspective , 1993, J. Softw. Maintenance Res. Pract..

[78]  Erik Arisholm,et al.  Empirical assessment of the impact of structural properties on the changeability of object-oriented software , 2006, Inf. Softw. Technol..

[79]  Victor R. Basili,et al.  Q‐MOPP: qualitative evaluation of maintenance organizations, processes and products , 1998 .

[80]  Harvey P. Siy,et al.  Predicting Fault Incidence Using Software Change History , 2000, IEEE Trans. Software Eng..

[81]  Hasan Ural,et al.  Test generation based on control and data dependencies within system specifications in SDL , 2000, Comput. Commun..

[82]  Jonathan I. Maletic,et al.  Journal of Software Maintenance and Evolution: Research and Practice Survey a Survey and Taxonomy of Approaches for Mining Software Repositories in the Context of Software Evolution , 2022 .

[83]  W. Lam,et al.  Managing change in software development using a process improvement approach , 1998, Proceedings. 24th EUROMICRO Conference (Cat. No.98EX204).

[84]  N. Schneidewind,et al.  Towards an Ontology of software maintenance , 1999 .

[85]  Audris Mockus,et al.  Using Version Control Data to Evaluate the Impact of Software Tools: A Case Study of the Version Editor , 2002, IEEE Trans. Software Eng..

[86]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[87]  Nicolas Anquetil,et al.  Organizing the Knowledge Used in Software Maintenance , 2003, J. Univers. Comput. Sci..

[88]  Sakti P. Ghosh.  Statistical relational tables for statistical database management , 1986, IEEE Transactions on Software Engineering.

[89]  Donald C. Pelz,et al.  Innovation Complexity and the Sequence of Innovating Stages , 1985 .

[90]  Dieter Hogrefe,et al.  The CCITT-Specification and Description Language SDL , 1989, Comput. Networks.

[91]  Celeste See Pui Ng,et al.  Journal of Software Maintenance and Evolution: Research and Practice a Decision Framework for Enterprise Resource Planning Maintenance and Upgrade: a Client Perspective , 2022 .

[92]  E. Burton Swanson,et al.  Characteristics of application software maintenance , 1978, CACM.

[93]  Daniel M. Germán,et al.  What do large commits tell us?: a taxonomical study of large commits , 2008, MSR '08.

[94]  Dewayne E. Perry,et al.  Software Faults in Evolving a Large, Real-Time System: a Case Study , 1993, ESEC.

[95]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[96]  Magne Jørgensen,et al.  Experience With the Accuracy of Software Maintenance Task Effort Prediction Models , 1995, IEEE Trans. Software Eng..

[97]  Adam A. Porter,et al.  Empirical studies of software engineering: a roadmap , 2000, ICSE '00.

[98]  Susan P. Williams,et al.  Characterising Requirements Volatility: An Empirical Case Study , 2005, ISESE.

[99]  Kent L. Beck,et al.  Embracing Change with Extreme Programming , 1999, Computer.

[100]  Yuan-Chuen Lin,et al.  An Executable Protocol Test Sequence Generation Method for EFSM-specified Protocols , 1996 .

[101]  Hans van Vliet,et al.  Two case studies in measuring software maintenance effort , 1998, Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272).

[102]  Michael Q. Rieck,et al.  Distributed routing schemes for ad hoc networks using d-SPR sets , 2004, Microprocess. Microsystems.