A fine-grained requirement traceability evolutionary algorithm: Kromaia, a commercial video game case study

Abstract Context:Commercial video games usually feature an extensive source code and requirements that are related to code lines from multiple methods. Traceability is vital in terms of maintenance and content update, so it is necessary to explore such search spaces properly. Objective:This work presents and evaluates CODFREL (Code Fragment-based Requirement Location), our approach to fine-grained requirement traceability, which lies in an evolutionary algorithm and includes encoding and genetic operators to manipulate code fragments that are built from source code lines. We compare it with a baseline approach (Regular-LSI) by configuring both approaches with different granularities (code lines / complete methods). Method:We evaluated our approach and Regular-LSI in the Kromaia video game case study, which is a commercial video game released on PC and PlayStation 4. The approaches are configured with method and code line granularity and work on 20 requirements that are provided by the development company. Our approach and Regular-LSI calculate similarities between requirements and code fragments or methods to propose possible solutions and, in the case of CODFREL, to guide the evolutionary algorithm. Results:The results, which compare code line and method granularity configurations of CODFREL with different granularity configurations of Regular-LSI, show that our approach outperforms Regular-LSI in precision and recall, with values that are 26 and 8 times better, respectively, even though it does not achieve the optimal solutions. We make an open-source implementation of CODFREL available. Conclusions:Since our approach takes into consideration key issues like the source code size in commercial video games and the requirement dispersion, it provides better starting points than Regular-LSI in the search for solution candidates for the requirements. However, the results and the influence of domain-specific language on them show that more explicit knowledge is required to improve such results.

[1]  Jane Cleland-Huang,et al.  Improving automated requirements trace retrieval: a study of term-based enhancement methods , 2010, Empirical Software Engineering.

[2]  Gordon Fraser,et al.  Parameter tuning or default values? An empirical investigation in search-based software engineering , 2013, Empirical Software Engineering.

[3]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[4]  Bogdan Dit,et al.  Using Data Fusion and Web Mining to Support Feature Location in Software , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[5]  Genny Tortora,et al.  Assessing IR-based traceability recovery tools through controlled experiments , 2009, Empirical Software Engineering.

[6]  Genny Tortora,et al.  Recovering traceability links in software artifact management systems using information retrieval methods , 2007, TSEM.

[7]  Genny Tortora,et al.  ADAMS Re-Trace: a traceability recovery tool , 2005, Ninth European Conference on Software Maintenance and Reengineering.

[8]  Mordechai Nisenson,et al.  A Traceability Technique for Specifications , 2008, 2008 16th IEEE International Conference on Program Comprehension.

[9]  Andrea De Lucia,et al.  COCONUT: COde COmprehension Nurturant Using Traceability , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[10]  Jane Huffman Hayes,et al.  Application of Swarm Techniques to Requirements Engineering: Requirements Tracing , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[11]  Tony Gorschek,et al.  Large-scale information retrieval in software engineering - an experience report from industrial application , 2016, Empirical Software Engineering.

[12]  Bogdan Dit,et al.  Feature location in source code: a taxonomy and survey , 2013, J. Softw. Evol. Process..

[13]  Andrea De Lucia,et al.  Traceability Recovery Using Numerical Analysis , 2009, 2009 16th Working Conference on Reverse Engineering.

[14]  George Spanoudakis,et al.  Software Traceability : A Roadmap , 2005 .

[15]  Arbi Ghazarian.  A Research Agenda for Software Reliability , 2009 .

[16]  Jaime Font,et al.  Evolutionary Algorithm for Bug Localization in the Reconfigurations of Models at Runtime , 2018, MoDELS.

[17]  Denys Poshyvanyk,et al.  Feature location via information retrieval based filtering of a single scenario execution trace , 2007, ASE.

[18]  Ricardo Baeza-Yates,et al.  Modern Information Retrieval - the concepts and technology behind search, Second edition , 2011 .

[19]  Patrick Mäder,et al.  Preventing Defects: The Impact of Requirements Traceability Completeness on Software Quality , 2017, IEEE Transactions on Software Engineering.

[20]  Mark Neal,et al.  Why and how of requirements tracing , 1994, IEEE Software.

[21]  Gabriele Bavota,et al.  Enhancing software artefact traceability recovery processes with link count information , 2014, Inf. Softw. Technol..

[22]  Andrea De Lucia,et al.  On the role of the nouns in IR-based traceability recovery , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[23]  Marsha Chechik,et al.  A Survey of Feature Location Techniques , 2013, Domain Engineering, Product Lines, Languages, and Conceptual Models.

[24]  Olly Gotel,et al.  An analysis of the requirements traceability problem , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[25]  Jane Cleland-Huang,et al.  Semantically Enhanced Software Traceability Using Deep Learning Techniques , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[26]  Bogdan Dit.  Configuring and Assembling Information Retrieval Based Solutions for Software Engineering Tasks , 2016, 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[27]  Jane Huffman Hayes,et al.  Technique Integration for Requirements Assessment , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[28]  Jaime Font,et al.  Leveraging variability modeling to address metamodel revisions in Model-based Software Product Lines , 2017, Comput. Lang. Syst. Struct..

[29]  Per Runeson,et al.  Recovering from a decade: a systematic mapping of information retrieval approaches to software traceability , 2013, Empirical Software Engineering.

[30]  Stephan M. Winkler,et al.  Genetic Algorithms and Genetic Programming - Modern Concepts and Practical Applications , 2009 .

[31]  Jane Cleland-Huang,et al.  Model-based traceability , 2009, 2009 ICSE Workshop on Traceability in Emerging Forms of Software Engineering.

[32]  Jane Cleland-Huang,et al.  Poirot: A Distributed Tool Supporting Enterprise-Wide Automated Traceability , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[33]  Jos Nijhuis,et al.  Design Decisions: The Bridge between Rationale and Architecture , 2006 .

[34]  Stephen V. Stehman,et al.  Selecting and interpreting measures of thematic classification accuracy , 1997 .

[35]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[36]  Jane Cleland-Huang,et al.  Towards an intelligent domain-specific traceability solution , 2014, ASE.

[37]  Yann-Gaël Guéhéneuc,et al.  Feature Location Using Probabilistic Ranking of Methods Based on Execution Scenarios and Information Retrieval , 2007, IEEE Transactions on Software Engineering.

[38]  Andrea De Lucia,et al.  On integrating orthogonal information retrieval methods to improve traceability recovery , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[39]  Bintu G. Vasudevan,et al.  A Comparative Study of Document Correlation Techniques for Traceability Analysis , 2008, ICEIS.