Automated Requirements Traceability: The Study of Human Analysts

The requirements traceability matrix (RTM) supports many software engineering and software verification and validation (V&V) activities such as change impact analysis, reverse engineering, reuse, and regression testing. The generation of RTMs is tedious and error-prone, though, thus RTMs are often not generated or maintained. Automated techniques have been developed to generate candidate RTMs with some success. When using RTMs to support the V&V of mission-or safety-critical systems, however, a human analyst must vet the candidate RTMs. The focus thus becomes the quality of the final RTM. This paper investigate show human analysts perform when vetting candidate RTMs. Specifically, a study was undertaken at two universities and had 26 participants analyze RTMs of varying accuracy for a Java code formatter program. The study found that humans tend to move their candidate RTM toward the line that represents recall = precision. Participants who examined RTMs with low recall and low precision drastically improved both.

[1]  Stefan Biffl,et al.  A case study on value-based requirements tracing , 2005, ESEC/FSE-13.

[2]  Oussama Ben Khadra,et al.  Goal-centric traceability for managing non-functional requirements , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[3]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[4]  Jane Cleland-Huang,et al.  Clustering support for automated tracing , 2007, ASE '07.

[5]  Jane Cleland-Huang,et al.  Phrasing in Dynamic Requirements Trace Retrieva , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[6]  Jane Cleland-Huang,et al.  Poirot: A Distributed Tool Supporting Enterprise-Wide Automated Traceability , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[7]  Carl K. Chang,et al.  Automating performance-related impact analysis through event based traceability , 2003, Requirements Engineering.

[8]  Kenneth Hugdahl,et al.  Prediction of human errors by maladaptive changes in event-related brain networks , 2008, Proceedings of the National Academy of Sciences.

[9]  Olly Gotel,et al.  An analysis of the requirements traceability problem , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[10]  Jane Huffman Hayes,et al.  Humans in the traceability loop: can't live with 'em, can't live without 'em , 2005, TEFSE '05.

[11]  Jane Huffman Hayes,et al.  Text mining for software engineering: how analyst feedback impacts final results , 2005, MSR '05.

[12]  Giuliano Antoniol,et al.  Recovering Traceability Links between Code and Documentation , 2002, IEEE Trans. Software Eng..

[13]  Emma Greenwood,et al.  Tumour suppressors: A new class? , 2001, Nature Reviews Cancer.

[14]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[15]  Alain April,et al.  REquirements TRacing On target (RETRO): improving software maintenance through traceability recovery , 2007, Innovations in Systems and Software Engineering.

[16]  Jane Cleland-Huang,et al.  Goal-Centric Traceability: Using Virtual Plumblines to Maintain Critical Systemic Qualities , 2008, IEEE Transactions on Software Engineering.

[17]  Tore Dybå,et al.  Conducting realistic experiments in software engineering , 2002, Proceedings International Symposium on Empirical Software Engineering.

[18]  Andrian Marcus,et al.  Recovering documentation-to-source-code traceability links using latent semantic indexing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[19]  Ofer Zwikael,et al.  Starting a New Project , 2011 .

[20]  Jane Huffman Hayes,et al.  Tracing requirements to defect reports: an application of information retrieval techniques , 2005, Innovations in Systems and Software Engineering.

[21]  S. Robertson.  The probability ranking principle in IR , 1997 .

[22]  AntoniolGiuliano,et al.  Recovering Traceability Links between Code and Documentation , 2002 .

[23]  Jane Huffman Hayes,et al.  Helping analysts trace requirements: an objective look , 2004, Proceedings. 12th IEEE International Requirements Engineering Conference, 2004..

[24]  A. Dekhtyar,et al.  Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools? , 2007, Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007).

[25]  Jane Huffman Hayes,et al.  Advancing candidate link generation for requirements tracing: the study of methods , 2006, IEEE Transactions on Software Engineering.

[26]  Jane Huffman Hayes,et al.  Improving requirements tracing via information retrieval , 2003, Proceedings. 11th IEEE International Requirements Engineering Conference, 2003..