Standing on the Shoulders of Software Product Line Research for Testing Systems of Systems

The complex and dynamic nature of Systems of Systems (SoSs) poses many challenges on their validation and testing, but so far few effective test strategies exist to address them. On the other hand, extensive research has been conducted in the testing of Software Product Lines (SPLs), which present interesting convergence points with SoSs, as both disciplines aim at reducing development costs and time-to-market thanks to extensive reuse of existing artifacts. In this paper, we outline commonalities and differences between the SoS and SPL paradigms from the point of view of testing and investigate how existing methods and tools from SPL testing could be leveraged to address the challenges of SoS testing.

[1]  Nenad Medvidovic,et al.  Unmanned and Autonomous Systems of Systems Test and Evaluation: Challenges and Opportunities , 2010 .

[2]  Myungchul Kim,et al.  Towards a Formal Framework for Product Line Test Development , 2007, 7th IEEE International Conference on Computer and Information Technology (CIT 2007).

[3]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[4]  Ricardo Valerdi,et al.  Test and evaluation of a SoS using a prescriptive and adaptive testing framework , 2010, 2010 5th International Conference on System of Systems Engineering.

[5]  Ricardo Pineda,et al.  Basis Path Analysis for Testing Complex System of Systems , 2013, Complex Adaptive Systems.

[6]  Ricardo Pineda,et al.  Integration, Verification, Validation, Test, and Evaluation (IVVT&E) Framework for System of Systems (SoS) , 2013, Complex Adaptive Systems.

[7]  John D. McGregor,et al.  Strategies for testing products in software product lines , 2012, ACM SIGSOFT Softw. Eng. Notes.

[8]  BertolinoAntonia,et al.  Testing Relative to Usage Scope , 2020, ACM Trans. Softw. Eng. Methodol..

[9]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[10]  Juha Taina,et al.  Product family testing: a survey , 2004, SOEN.

[11]  J.S. Dahmann,et al.  Understanding the Current State of US Defense Systems of Systems and the Implications for Systems Engineering , 2008, 2008 2nd Annual IEEE Systems Conference.

[12]  Klaus Pohl,et al.  Testing Variabilities in Use Case Models , 2003, PFE.

[13]  Stuart Harvey Rubin,et al.  Randomization for testing systems of systems , 2009, 2009 IEEE International Conference on Information Reuse & Integration.

[14]  Francesca Lonetti,et al.  Governing Regression Testing in Systems of Systems , 2019, 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[15]  Per Runeson,et al.  Software product line testing - A systematic mapping study , 2011, Inf. Softw. Technol..

[16]  Antonia Bertolino,et al.  Do We Need New Strategies for Testing Systems-of-Systems? , 2018, 2018 IEEE/ACM 6th International Workshop on Software Engineering for Systems-of-Systems (SESoS).

[17]  Sérgio Soares,et al.  The Role of Rapid Reviews in Supporting Decision-Making in Software Engineering Practice , 2018, EASE.

[18]  Mario Piattini,et al.  Software Product Line Testing - A Systematic Review , 2009, ICSOFT.

[19]  Arnaud Gotlieb,et al.  PACOGEN: Automatic Generation of Pairwise Test Configurations from Feature Models , 2011, 2011 IEEE 22nd International Symposium on Software Reliability Engineering.

[20]  Alexander Egyed,et al.  A systematic mapping study of search-based software engineering for software product lines , 2015, Inf. Softw. Technol..

[21]  John D. McGregor,et al.  A systematic mapping study of software product lines testing , 2011, Inf. Softw. Technol..

[22]  Antonia Bertolino,et al.  Use case-based testing of product lines , 2003, ESEC/FSE-11.

[23]  Clémentine Nebut,et al.  System Testing of Product Lines: From Requirements to Test Cases , 2006, Software Product Lines.

[24]  Dragan Gasevic,et al.  Evolutionary Search-Based Test Generation for Software Product Line Feature Models , 2012, CAiSE.

[25]  Avelino F. Zorzo,et al.  Model-based testing of software product lines: Mapping study and research roadmap , 2020, J. Syst. Softw..

[26]  Mark W. Maier Architecting Principles for Systems‐of‐Systems , 1996 .

[27]  Francesca Lonetti,et al.  EDUFYSoS: A Factory of Educational System of Systems Case Studies , 2020, 2020 IEEE 15th International Conference of System of Systems Engineering (SoSE).

[28]  Paola Inverardi,et al.  Changing Software in a Changing World: How to Test in Presence of Variability, Adaptation and Evolution? , 2019, From Software Engineering to Formal Methods and Tools, and Back.

[29]  John D. McGregor,et al.  On strategies for testing software product lines: A systematic literature review , 2014, Inf. Softw. Technol..

[30]  Sungwon Kang,et al.  A survey on software product line testing , 2012, SPLC '12.

[31]  Sergio Segura,et al.  A Comparison of Test Case Prioritization Criteria for Software Product Lines , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[32]  Ilaria Matteucci,et al.  Addressing Security Properties in Systems of Systems: Challenges and Ideas , 2019, SERENE.