Using requirements and design information to predict volatility in software development

We hypothesise that data about the requirements and design stages of a software development project can be used to make predictions about the subsequent number of development changes that software components will experience. This would allow managers to concentrate time-consuming efforts (such as traceability and staff training) to a few at-risk, cost-effective areas, and may also allow predictions to be made at an earlier stage than is possible using traditional metrics, such as lines of code. Previous researchers have studied links between change-proneness and metrics such as measures of inheritance, size and code coupling. We extend these studies by including measures of requirements and design activity as well. Firstly we develop structures to model the requirements and design processes, and then propose some new metrics based on these models. The structures are populated using data from a case study project and analysed alongside existing complexity metrics to ascertain whether change-proneness can be predicted. Finally we examine whether combining these metrics with existing metrics improves our ability to make predictions about change-proneness. First results show that our metrics can be linked to the quantity of change experienced by components in a software development project (potentially allowing predictions to take place earlier than before) but that best results are obtained by combining existing complexity metrics such as size, or combining existing metrics with our newer metrics.

[1]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[2]  Andreas Zeller,et al.  Mining version histories to guide software changes , 2005, Proceedings. 26th International Conference on Software Engineering.

[3]  C. van Koten,et al.  An application of Bayesian network for predicting object-oriented software maintainability , 2006, Inf. Softw. Technol..

[4]  Olly Gotel,et al.  Extended requirements traceability: results of an industrial case study , 1997, Proceedings of ISRE '97: 3rd IEEE International Symposium on Requirements Engineering.

[5]  David P. Darcy,et al.  Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis , 1998, IEEE Trans. Software Eng..

[6]  Richard C. Holt,et al.  Predicting change propagation in software systems , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[7]  Matthias Jarke,et al.  Toward Reference Models of Requirements Traceability , 2001, IEEE Trans. Software Eng..

[8]  Meir M. Lehman,et al.  Program evolution , 1984, Inf. Process. Manag..

[9]  W. Lam,et al.  Managing change in software development using a process improvement approach , 1998, Proceedings. 24th EUROMICRO Conference (Cat. No.98EX204).

[10]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[11]  Balasubramaniam Ramesh,et al.  Requirements traceability: Theory and practice , 1997, Ann. Softw. Eng..

[12]  Keith H. Bennett,et al.  Software maintenance and evolution: a roadmap , 2000, ICSE '00.

[13]  Vijay K. Vaishnavi,et al.  Predicting Maintenance Performance Using Object-Oriented Design Complexity Metrics , 2003, IEEE Trans. Software Eng..

[14]  Ian Sommerville,et al.  Requirements Engineering: Processes and Techniques , 1998 .

[15]  Balasubramaniam Ramesh,et al.  Implementing requirements traceability: a case study , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[16]  Stefan Biffl,et al.  Value-Based Requirements Traceability: Lessons Learned , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[17]  Paul Clements,et al.  The Inevitable Pain of Software Development , Including of Extreme Programming , Caused by Requirements Volatility , .

[18]  Lionel C. Briand,et al.  A Unified Framework for Coupling Measurement in Object-Oriented Systems , 1999, IEEE Trans. Software Eng..

[19]  Rudolf K. Keller,et al.  Cohesion as changeability indicator in object-oriented systems , 2001, Proceedings Fifth European Conference on Software Maintenance and Reengineering.

[20]  Carl K. Chang,et al.  Event-Based Traceability for Managing Evolutionary Change , 2003, IEEE Trans. Software Eng..

[21]  Stan Matwin,et al.  Mining the Software Change Repository of a Legacy Telephony System , 2004, MSR.

[22]  Ned Chapin,et al.  Types of software evolution and software maintenance , 2001, J. Softw. Maintenance Res. Pract..

[23]  Eleni Stroulia,et al.  Data-mining in Support of Detecting Class Co-evolution , 2004, SEKE.

[24]  Martin J. Shepperd,et al.  Estimating Software Project Effort Using Analogies , 1997, IEEE Trans. Software Eng..

[25]  Charles J. Petrie,et al.  Constrained Decision Revision , 1992, AAAI.

[26]  David A. Gustafson,et al.  Software measure specification , 1993, SIGSOFT '93.

[27]  Barry W. Boehm,et al.  Value-Based Software Engineering: A Case Study , 2003, Computer.

[28]  Mario Piattini,et al.  A Metric-Based Approach for Predicting Conceptual Data Models Maintainability , 2001, Int. J. Softw. Eng. Knowl. Eng..

[29]  John A. McDermid,et al.  Safety Case Construction and Reuse Using Patterns , 1997, SAFECOMP.

[30]  Olly Gotel,et al.  An analysis of the requirements traceability problem , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[31]  Fred P. Brooks,et al.  The Mythical Man-Month , 1975, Reliable Software.

[32]  Richard H. Carver,et al.  An Evaluation of the MOOD Set of Object-Oriented Software Metrics , 1998, IEEE Trans. Software Eng..

[33]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[34]  Yijun Yu,et al.  Analyzing the evolution of eclipse plugins , 2008, MSR '08.

[35]  M. R. Strens,et al.  Change analysis: a step towards meeting the challenge of changing requirements , 1996, Proceedings IEEE Symposium and Workshop on Engineering of Computer-Based Systems.

[36]  Giuliano Antoniol,et al.  Estimating the size of changes for evolving object oriented systems: a case study , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[37]  Rüdiger Lincke,et al.  Comparing software metrics tools , 2008, ISSTA '08.

[38]  Leszek A. Maciaszek,et al.  Requirements Analysis and System Design , 2001 .

[39]  Vasant Dhar,et al.  Supporting Systems Development by Capturing Deliberations During Requirements Engineering , 1992, IEEE Trans. Software Eng..

[40]  Harald C. Gall,et al.  Visualizing multiple evolution metrics , 2005, SoftVis '05.

[41]  E. Burton Swanson,et al.  Characteristics of application software maintenance , 1978, CACM.

[42]  Doo-Hwan Bae,et al.  Behavioral Dependency Measurement for Change-Proneness Prediction in UML 2.0 Design Models , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[43]  Jane Cleland-Huang,et al.  Goal-centric traceability for managing non-functional requirements , 2005, ICSE.

[44]  Suzanne Robertson,et al.  Mastering the Requirements Process , 1999 .

[45]  Norman E. Fenton,et al.  Software metrics: roadmap , 2000, ICSE '00.

[46]  Laszlo A. Belady Modifiability of large software systems , 1980, Operating Systems Engineering.

[47]  Steve Riddle,et al.  Tailoring Traceability Information to Business Needs , 2006 .

[48]  Natalia Juristo Juzgado,et al.  Is the European Industry Moving toward Solving Requirements Engineering Problems? , 2002, IEEE Softw..

[49]  BoehmBarry,et al.  Using the WinWin Spiral Model , 1998 .

[50]  M. Wermelinger,et al.  Using Formal Concept Analysis to Construct and Visualise Social Hierarchies of Software Developers , 2009 .

[51]  Brian R. Huguenard,et al.  Evaluating Aptness of a Regression Model , 2007 .

[52]  Elaine J. Weyuker,et al.  Comments on "Toward a Framework for Software Measurement Validation" , 1997, IEEE Trans. Software Eng..

[53]  Terri Gullickson The Psychology Research Handbook: A Guide for Graduate Students and Research Assistants. , 1997 .

[54]  Carl H. Smith,et al.  On Weyuker's Axioms For Software Complexity Measures , 1991, IEEE Trans. Software Eng..

[55]  Yann-Gaël Guéhéneuc,et al.  Extracting Change-patterns from CVS Repositories , 2006, 2006 13th Working Conference on Reverse Engineering.

[56]  Dewayne E. Perry,et al.  Metrics and laws of software evolution-the nineties view , 1997, Proceedings Fourth International Software Metrics Symposium.

[57]  Barry W. Boehm,et al.  A spiral model of software development and enhancement , 1986, Computer.

[58]  Didar Zowghi,et al.  A study of the impact of requirements volatility on software project performance , 2002, Ninth Asia-Pacific Software Engineering Conference, 2002..

[59]  Eric S. K. Yu,et al.  Models for supporting the redesign of organizational work , 1995, COCS '95.

[60]  Jane Cleland-Huang,et al.  The Detection and Classification of Non-Functional Requirements with Application to Early Aspects , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[61]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[62]  Sallie M. Henry,et al.  Quantitative assessment of the software maintenance process and requirements volatility , 1993, CSC '93.

[63]  Barry W. Boehm,et al.  Requirements that Handle IKIWISI, COTS, and Rapid Change , 2000, Computer.

[64]  Stephen Fickas,et al.  Goal-Directed Requirements Acquisition , 1993, Sci. Comput. Program..

[65]  Audris Mockus,et al.  Identifying reasons for software changes using historic databases , 2000, Proceedings 2000 International Conference on Software Maintenance.

[66]  Hugh Coolican Research Methods and Statistics in Psychology , 1993 .

[67]  Jane Cleland-Huang,et al.  A heterogeneous solution for improving the return on investment of requirements traceability , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[68]  Tom Gilb,et al.  Evolutionary development , 1981, SOEN.

[69]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[70]  Cem Kaner,et al.  Software Engineering Metrics: What Do They Measure and How Do We Know? , 2004 .

[71]  Lionel C. Briand,et al.  Dynamic coupling measurement for object-oriented software , 2004, IEEE Transactions on Software Engineering.

[72]  Elaine L. May,et al.  The Evolutionary Development Model for Software , 1996 .

[73]  Geert Poels,et al.  Distance-based software measurement: necessary and sufficient properties for software measures , 2000, Inf. Softw. Technol..

[74]  Stefan Biffl,et al.  A case study on value-based requirements tracing , 2005, ESEC/FSE-13.

[75]  Mikael Lindvall Evaluating Impact Analysis – A Case Study , 2004, Empirical Software Engineering.

[76]  Barbara A. Kitchenham,et al.  Coupling measures and change ripples in C++ application software , 2000, J. Syst. Softw..

[77]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[78]  Meir M. Lehman,et al.  An Introduction to growth dynamics , 1972, Statistical Computer Performance Evaluation.

[79]  P. Clarkson,et al.  Predicting change propagation in complex design , 2004 .

[80]  C. Kemerer,et al.  OO Metrics in Practice , 2005, IEEE Softw..

[81]  Mark Klein Capturing Design Rationale for CE Teams Mark Klein-1-Capturing Design Rationale in Concurrent Engineering Teams , 1993 .

[82]  Klaus Pohl,et al.  Process-Centered Requirements Engineering , 1996 .

[83]  Eric S. K. Yu,et al.  Towards modelling and reasoning support for early-phase requirements engineering , 1997, Proceedings of ISRE '97: 3rd IEEE International Symposium on Requirements Engineering.

[84]  Daniel M. Germán,et al.  An empirical study of fine-grained software modifications , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[85]  Didar Zowghi,et al.  Analysis of requirements volatility during software development life cycle , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[86]  C. Robson,et al.  Real World Research: A Resource for Social Scientists and Practitioner-Researchers , 1993 .

[87]  George Hripcsak,et al.  Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval , 2005, J. Am. Medical Informatics Assoc..

[88]  Elaine J. Weyuker,et al.  Evaluating Software Complexity Measures , 2010, IEEE Trans. Software Eng..

[89]  Tim Kelly,et al.  The Goal Structuring Notation – A Safety Argument Notation , 2004 .

[90]  Meir M. Lehman,et al.  Program evolution and its impact on software engineering , 1976, ICSE '76.

[91]  Alan F. Blackwell,et al.  Cognitive Dimensions of Information Artefacts: a tutorial , 1998 .

[92]  Lionel C. Briand,et al.  Using coupling measurement for impact analysis in object-oriented systems , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[93]  S. Jones,et al.  Was There a Hawthorne Effect? , 1992, American Journal of Sociology.

[94]  Harald C. Gall,et al.  Visualizing software release histories: the use of color and third dimension , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[95]  Kuntz Werner,et al.  Issues as Elements of Information Systems , 1970 .

[96]  Balasubramaniam Ramesh,et al.  Factors influencing requirements traceability practice , 1998, CACM.

[97]  Harvey Siy,et al.  If your ver-sion control system could talk , 1997 .

[98]  M.M. Lehman,et al.  Programs, life cycles, and laws of software evolution , 1980, Proceedings of the IEEE.

[99]  Thomas P. Moran,et al.  Questions, Options, and Criteria: Elements of Design Space Analysis , 1991, Hum. Comput. Interact..

[100]  E. James Whitehead,et al.  Identification of software instabilities , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[101]  H. J. Arnold Introduction to the Practice of Statistics , 1990 .

[102]  O. Gotel,et al.  Contribution Structures , 1995 .

[103]  E. Burton Swanson,et al.  The dimensions of maintenance , 1976, ICSE '76.

[104]  Neville Churcher,et al.  Comments on "A Metrics Suite for Object Oriented Design" , 1995, IEEE Trans. Software Eng..

[105]  David Lorge Parnas,et al.  Software aging , 1994, Proceedings of 16th International Conference on Software Engineering.

[106]  Miguel Goulão,et al.  Toward the Design Quality Evaluation of Object-Oriented Software Systems , 1995 .

[107]  Steve Easterbrook,et al.  Handling conflict between domain descriptions with computer-supported negotiation , 1991 .

[108]  S.D.P. Harker,et al.  The change and evolution of requirements as a challenge to the practice of software engineering , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[109]  Victor R. Basili,et al.  A Methodology for Collecting Valid Software Engineering Data , 1984, IEEE Transactions on Software Engineering.

[110]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[111]  Sandro Morasca,et al.  Property-Based Software Engineering Measurement , 1996, IEEE Trans. Software Eng..

[112]  Harald C. Gall,et al.  Mining Software Evolution to Predict Refactoring , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[113]  Wei Li,et al.  Object-Oriented Metrics Which Predict Maintainability , 1993 .

[114]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[115]  Jane Cleland-Huang Just Enough Requirements Traceability , 2006, COMPSAC.

[116]  Barry W. Boehm,et al.  How Much Software Quality Investment Is Enough: A Value-Based Approach , 2006, IEEE Software.

[117]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[118]  Matthias Jarke,et al.  Requirements tracing , 1998, CACM.

[119]  René Bañares-Alcántara,et al.  Design support systems for process engineering—II. KBDS: An experimental prototype , 1995 .

[120]  John C. Windsor,et al.  Determinants of software volatility: a field study , 2003, J. Softw. Maintenance Res. Pract..

[121]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[122]  William C. Regli,et al.  A Survey of Design Rationale Systems: Approaches, Representation, Capture and Retrieval , 2000, Engineering with Computers.

[123]  Yijun Yu,et al.  Using formal concept analysis to construct and visualise hierarchies of socio-technical relations , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[124]  William M. Evanco,et al.  Comments on "The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics" , 2003, IEEE Trans. Software Eng..

[125]  Barry W. Boehm,et al.  Using the WinWin Spiral Model: A Case Study , 1998, Computer.

[126]  Khaled El Emam,et al.  The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics , 2001, IEEE Trans. Software Eng..

[127]  P. Sprent,et al.  19. Applied Nonparametric Statistical Methods , 1995 .

[128]  S.A. Bohner,et al.  Software change impacts-an evolving perspective , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[129]  George E. Stark,et al.  An examination of the effects of requirements changes on software maintenance releases , 1999, J. Softw. Maintenance Res. Pract..

[130]  Keith H. Bennett,et al.  Software evolution: past, present and future , 1996, Inf. Softw. Technol..

[131]  Tom Mens,et al.  Future trends in software evolution metrics , 2001, IWPSE '01.

[132]  Victor R. Basili,et al.  Experimentation in software engineering , 1986, IEEE Transactions on Software Engineering.

[133]  Stefan Biffl,et al.  Determining the cost-quality trade-off for automated software traceability , 2005, ASE.

[134]  Michael L. Begeman,et al.  gIBIS: A tool for all reasons , 1989, JASIS.

[135]  Jürgen Börstler,et al.  An industrial case study on requirements volatility measures , 2005, 12th Asia-Pacific Software Engineering Conference (APSEC'05).

[136]  John Mylopoulos,et al.  From object-oriented to goal-oriented requirements analysis , 1999, CACM.

[137]  Martin Loomes,et al.  Requirements evolution in the midst of environmental change: a managed approach , 1998, Proceedings of the Second Euromicro Conference on Software Maintenance and Reengineering.

[138]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[139]  Horst Zuse,et al.  A Framework of Software Measurement , 1998 .

[140]  Steve Riddle,et al.  Overcoming the traceability benefit problem , 2005, 13th IEEE International Conference on Requirements Engineering (RE'05).