On the gamification of human-centric traceability tasks in software testing and coding

Traceability is the ability to trace the influence of one software artifact on another by linking dependencies. Test-to-code traceability (relationships between test and system code) plays a vital role in the production, verification, reliability and certification of highly dependable systems. In practical settings, however, traceability tasks are most often observed in the breach by human developers/testers. Prior research works on test-to-code traceability in software engineering do not provide high traceability output and accuracy as they mainly rely on sought-after approaches to recover links. Gamified Software Engineering (GSE) is a growing field that in particular taps into gamification, the application of game mechanics in non-game contexts, to address human-related concerns in the field of SE. This paper argues that a new gamified approach is necessary to tackle the human issue of capturing traceability information in a by-product manner. Thus, it advocates for the induction of gamification concepts in software traceability. We propose a conceptual framework where the gamification is infusing engagement into human-centric traceability tasks to record trace links. An empirical evaluation was performed to assess the performance of the framework compared with a state-of-the-art approach.

[1]  Henry Muccini,et al.  A scoped approach to traceability management , 2009, J. Syst. Softw..

[2]  AntoniolGiuliano,et al.  Recovering Traceability Links between Code and Documentation , 2002 .

[3]  Christopher Kumar Anand,et al.  Gamification and serious game approaches for adult literacy tablet software , 2014, Entertain. Comput..

[4]  Chin-Yu Huang,et al.  Analysis of incorporating logistic testing-effort function into software reliability modeling , 2002, IEEE Trans. Reliab..

[5]  Juho Hamari,et al.  Demographic differences in perceived benefits from gamification , 2014, Comput. Hum. Behav..

[6]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[7]  Gabriele Bavota,et al.  SCOTCH: Test-to-code traceability using slicing and conceptual coupling , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[8]  Sai Peck Lee,et al.  Achievements and Challenges in State-of-the-Art Software Traceability Between Test and Code Artifacts , 2014, IEEE Transactions on Reliability.

[9]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[10]  Jens von Pilgrim,et al.  A survey of traceability in requirements engineering and model-driven development , 2010, Software & Systems Modeling.

[11]  Patrick Mäder,et al.  Software traceability: trends and future directions , 2014, FOSE.

[12]  Andrea De Lucia,et al.  Recovering traceability links between unit tests and classes under test: An improved method , 2010, 2010 IEEE International Conference on Software Maintenance.

[13]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[14]  Cécile Paris,et al.  Gamification for Online Communities: A Case Study for Delivering Government Services , 2014, Int. J. Cooperative Inf. Syst..

[15]  Arie van Deursen,et al.  The Video Store Revisited – Thoughts on Refactoring and Testing , 2002 .

[16]  Klaus Pohl,et al.  Adapting traceability environments to project-specific needs , 1998, CACM.

[17]  Giuliano Antoniol,et al.  Recovering Traceability Links between Code and Documentation , 2002, IEEE Trans. Software Eng..

[18]  Andrea Zisman,et al.  XTraQue: traceability for product line systems , 2009, Software & Systems Modeling.

[19]  Serge Demeyer,et al.  Establishing Traceability Links between Unit Test Cases and Units under Test , 2009, 2009 13th European Conference on Software Maintenance and Reengineering.

[20]  Jane Huffman Hayes,et al.  Assessing traceability of software engineering artifacts , 2010, Requirements Engineering.

[21]  Giordano Tamburrelli,et al.  Understanding gamification mechanisms for software development , 2013, ESEC/FSE 2013.

[22]  Mario Piattini,et al.  Gamification in software engineering - A systematic mapping , 2015, Inf. Softw. Technol..

[23]  Yonggang Zhang,et al.  Text mining and software engineering: an integrated source code and document analysis approach , 2008, IET Softw..

[24]  Gabriele Bavota,et al.  Recovering test-to-code traceability using slicing and textual analysis , 2014, J. Syst. Softw..

[25]  Enrico Zio,et al.  IEEE Reliability Society Technical Operations Annual Technical Report for 2010 , 2010, IEEE Transactions on Reliability.

[26]  John C. Grundy,et al.  Improving automated documentation to code traceability by combining retrieval techniques , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[27]  Xin Yao,et al.  Multi-Objective Approaches to Optimal Testing Resource Allocation in Modular Software Systems , 2010, IEEE Transactions on Reliability.

[28]  Mark Neal,et al.  Why and how of requirements tracing , 1994, IEEE Software.

[29]  Alexander Egyed,et al.  A Scenario-Driven Approach to Trace Dependency Analysis , 2003, IEEE Trans. Software Eng..

[30]  Bogdan Vasilescu,et al.  Human aspects, gamification, and social media in collaborative software engineering , 2014, ICSE Companion.

[31]  Yi Zhang,et al.  Strategic Traceability for Safety-Critical Projects , 2013, IEEE Software.

[32]  Rebeca P. Díaz Redondo,et al.  A social gamification framework for a K-6 learning platform , 2013, Comput. Hum. Behav..

[33]  Michael R. Lyu,et al.  Optimal testing resource allocation, and sensitivity analysis in software development , 2005, IEEE Transactions on Reliability.

[34]  Harry M. Sneed Reverse engineering of test cases for selective regression testing , 2004, Eighth European Conference on Software Maintenance and Reengineering, 2004. CSMR 2004. Proceedings..

[35]  Romain Robbes,et al.  Linking e-mails and source code artifacts , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[36]  Miguel Mira da Silva,et al.  iThink: A Game-Based Approach Towards Improving Collaboration and Participation in Requirement Elicitation , 2012, VS-GAMES.

[37]  Arbi Ghazarian A Research Agenda for Software Reliability , 2009 .

[38]  Shigeru Yamada,et al.  Optimal allocation and control problems for software-testing resources , 1990 .

[39]  Sai Peck Lee,et al.  A test-to-code traceability method using .NET custom attributes , 2015, RACS.

[40]  Andrian Marcus,et al.  Recovering documentation-to-source-code traceability links using latent semantic indexing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[41]  Michael R. Lyu,et al.  Framework for modeling software reliability, using various testing-efforts and fault-detection rates , 2001, IEEE Trans. Reliab..

[42]  Gabriele Bavota,et al.  Evaluating test‐to‐code traceability recovery methods through controlled experiments , 2013, J. Softw. Evol. Process..

[43]  Ning Chen GATE: game-based testing environment , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[44]  Alexander Schill,et al.  A Generic Platform for Enterprise Gamification , 2012, 2012 Joint Working IEEE/IFIP Conference on Software Architecture and European Conference on Software Architecture.

[45]  Genny Tortora,et al.  Recovering traceability links in software artifact management systems using information retrieval methods , 2007, TSEM.

[46]  Chao Liu,et al.  Recovering Relationships between Documentation and Source Code based on the Characteristics of Software Engineering , 2009, Electron. Notes Theor. Comput. Sci..

[47]  Luis de Marcos,et al.  An empirical study comparing gamification and social networking on e-learning , 2014, Comput. Educ..

[48]  Antonio Abad Civit Balcells,et al.  A game-based approach to the teaching of object-oriented programming languages , 2014, Comput. Educ..

[49]  Juan Garbajosa,et al.  A study to support agile methods more effectively through traceability , 2011, Innovations in Systems and Software Engineering.

[50]  Harvey S. Koch,et al.  Managing Test-Procedures to Achieve Reliable Software , 1983, IEEE Transactions on Reliability.

[51]  Daniel Nunan,et al.  Gamification and the online retail experience , 2014 .