Achieving traceability in large scale continuous integration and delivery deployment, usage and validation of the eiffel framework

The importance of traceability in software development has long been recognized, not only for reasons of legality and certification, but also to enable the development itself. At the same time, organizations are known to struggle to live up to traceability requirements, and there is an identified lack of studies on traceability practices in the industry, not least in the area of tooling and infrastructure. This paper presents, investigates and discusses Eiffel, an industry developed solution designed to provide real time traceability in continuous integration and delivery. The traceability needs of industry professionals are also investigated through interviews, providing context to that solution. It is then validated through further interviews, a comparison with previous traceability methods and a review of literature. It is found to address the identified traceability needs and found in some cases to reduce traceability data acquisition times from days to minutes, while at the same time alternatives offering comparable functionality are lacking. In this work, traceability is shown not only to be an important concern to engineers, but also regarded as a prerequisite to successful large scale continuous integration and delivery. At the same time, promising developments in technical infrastructure are documented and clear differences in traceability mindset between separate industry projects is revealed.

[1]  Patrick Mäder,et al.  Software traceability: trends and future directions , 2014, FOSE.

[2]  Roelf J. Wieringa,et al.  An Introduction to Requirements Traceability , 1995 .

[3]  Giuliano Antoniol,et al.  Recovering Traceability Links between Code and Documentation , 2002, IEEE Trans. Software Eng..

[4]  Hakan Erdogmus,et al.  CodeAware: Sensor-Based Fine-Grained Monitoring and Management of Software Artifacts , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[5]  Yuhoon Ki,et al.  An Open Source-Based Approach to Software Development Infrastructures , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[6]  Yingbo Wu,et al.  Continuously delivered? Periodically updated? Never changed? Studying an open source project's releases of code, requirements, and trace matrix , 2015, 2015 IEEE Workshop on Just-In-Time Requirements Engineering (JITRE).

[7]  Hoyt Lougee,et al.  SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION , 2001 .

[8]  L. M. Sherer,et al.  RADIO TECHNICAL COMMISSION FOR AERONAUTICS , 1947 .

[9]  Olly Gotel,et al.  An analysis of the requirements traceability problem , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[10]  Ilka Philippow,et al.  Motivation Matters in the Traceability Trenches , 2009, 2009 17th IEEE International Requirements Engineering Conference.

[11]  Brian Fitzgerald,et al.  The Transformation of Open Source Software , 2006, MIS Q..

[12]  Colin Robson,et al.  Real world research : a resource for users of social research methods in applied settings , 2011 .

[13]  Jan Bosch,et al.  Modeling continuous integration practice differences in industry software development , 2014, J. Syst. Softw..

[14]  Bente Anda,et al.  Experiences from conducting semi-structured interviews in empirical software engineering research , 2005, 11th IEEE International Software Metrics Symposium (METRICS'05).

[15]  Paul Clements,et al.  Software product lines - practices and patterns , 2001, SEI series in software engineering.

[16]  Matthias Jarke,et al.  Toward Reference Models of Requirements Traceability , 2001, IEEE Trans. Software Eng..

[17]  Jan Bosch,et al.  EXPERIENCED BENEFITS OF CONTINUOUS INTEGRATION IN INDUSTRY SOFTWARE PRODUCT DEVELOPMENT: A CASE STUDY , 2013, ICSE 2013.

[18]  George Mangalaraj,et al.  Challenges of migrating to agile methodologies , 2005, CACM.

[19]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[20]  Mike Roberts Enterprise Continuous Integration Using Binary Dependencies , 2004, XP.

[21]  Robert Dabrowski,et al.  One Graph to Rule Them All - Software Measurment and Management , 2012, CS&P.

[22]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[23]  Klaus Schmid,et al.  Software product lines in action - the best industrial practice in product line engineering , 2007 .

[24]  Bertrand Meyer,et al.  Eiffel: A language and environment for software engineering , 1988, J. Syst. Softw..

[25]  Ilka Philippow,et al.  A Survey on Usage Scenarios for Requirements Traceability in Practice , 2013, REFSQ.

[26]  Robert Dabrowski,et al.  Improving software quality by improving architecture management , 2012, CompSysTech '12.

[27]  Richard N. Taylor,et al.  Capturing custom link semantics among heterogeneous artifacts and tools , 2009, 2009 ICSE Workshop on Traceability in Emerging Forms of Software Engineering.

[28]  Richard N. Taylor,et al.  Software traceability with topic modeling , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[29]  Patrick Mäder,et al.  An empirical study on project-specific traceability strategies , 2013, 2013 21st IEEE International Requirements Engineering Conference (RE).

[30]  Barrie Dale,et al.  Quality Management Systems , 2015 .

[31]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[32]  Andrea De Lucia,et al.  On the Equivalence of Information Retrieval Methods for Automated Traceability Link Recovery , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[33]  Sue J. Waite,et al.  Real world research: a resource for users of social research methods in applied settings (3rd ed.) , 2012 .

[34]  Klaus Pohl,et al.  Adapting traceability environments to project-specific needs , 1998, CACM.

[35]  Jan Bosch,et al.  Continuous Integration and Delivery Traceability in Industry: Needs and Practices , 2016, 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[36]  Jan Bosch,et al.  Automated software integration flows in industry: a multiple-case study , 2014, ICSE Companion.

[37]  Jez Humble,et al.  Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation , 2010 .