Governing Regression Testing in Systems of Systems

Great advances in network technology and software engineering have triggered the development and spread of Systems of Systems (SoSs). The dynamic and evolvable nature of SoSs poses important challenges on the validation of such systems and in particular on their regression testing, aiming at assessing that run-time changes and evolutions do not introduce regression in SoS behavior. This paper outlines issues and challenges of regression testing of SoSs, identifying the main kinds of evolution that can impact on their regression testing activity. Furthermore, it presents a conceptual framework for governing the regression testing of SoSs. The proposed framework leverages the concept of an orchestration graph that describes the flow of test cases and sketches a solution for deriving a regression test plan according to test cases dependencies.

[1]  Mark W. Maier.  Architecting Principles for Systems‐of‐Systems , 1996 .

[2]  Antonia Bertolino,et al.  Do We Need New Strategies for Testing Systems-of-Systems? , 2018, 2018 IEEE/ACM 6th International Workshop on Software Engineering for Systems-of-Systems (SESoS).

[3]  Russ Abbott.  Open at the top; open at the bottom; and continually (but slowly) evolving , 2006, 2006 IEEE/SMC International Conference on System of Systems Engineering.

[4]  Andrea Polini,et al.  Testing software components for integration: a survey of issues and techniques , 2007, Softw. Test. Verification Reliab..

[5]  Rui Abreu,et al.  An Empirical Study on the Use of Defect Prediction for Test Case Prioritization , 2019, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[6]  Wasif Afzal,et al.  A Pragmatic Perspective on Regression Testing Challenges , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[7]  Diomidis Spinellis.  State-of-the-Art Software Testing , 2017, IEEE Softw..

[8]  Dayang N. A. Jawawi,et al.  Test case prioritization approaches in regression testing: A systematic literature review , 2018, Inf. Softw. Technol..

[9]  Marcelo d'Amorim,et al.  Time-space efficient regression testing for configurable systems , 2017, J. Syst. Softw..

[10]  Gregg Rothermel,et al.  Techniques for improving regression testing in continuous integration development environments , 2014, SIGSOFT FSE.

[11]  Andrea Polini,et al.  Enhancing Service Federation Trustworthiness through Online Testing , 2012, Computer.

[12]  Jan Peleska,et al.  Systems of Systems Engineering , 2015 .

[13]  Jan Tretmans,et al.  Model-Based Testing of Environmental Conformance of Components , 2006, FMCO.

[14]  Kai Petersen,et al.  Testing highly complex system of systems: An industrial case study , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[15]  Francesca Lonetti,et al.  A Proposal to Orchestrate Test Cases , 2018, 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC).

[16]  Radziah Mohamad,et al.  Effective Regression Test Case Selection , 2017, ACM Comput. Surv..

[17]  A. Orso,et al.  Retesting software during development and maintenance , 2008, 2008 Frontiers of Software Maintenance.

[18]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[19]  Antonello Calabrò,et al.  Monitoring Service Choreographies from Multiple Sources , 2012, SERENE.

[20]  Antonello Calabrò,et al.  A Generative Approach for the Adaptive Monitoring of SLA in Service Choreographies , 2013, ICWE.

[21]  Antonello Calabrò,et al.  Adaptive SLA monitoring of service choreographies enacted on the Cloud , 2013, 2013 IEEE 7th International Symposium on the Maintenance and Evolution of Service-Oriented and Cloud-Based Systems.

[22]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.

[23]  Ali Mesbah,et al.  Fine-Grained Test Minimization , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[24]  Jo Ann Lane,et al.  Systems of systems test and evaluation challenges , 2010, 2010 5th International Conference on System of Systems Engineering.