Do We Need New Strategies for Testing Systems-of-Systems?

This paper overviews the main Systems-of-Systems (SoS) characteristics that can impact on their verification, validation and testing (VV&T). Furthermore, it addresses technical, conceptual, social and organizational challenges, discusses which existing approaches of VV&T can be used for SoS, and points out future research in the field.

[1]  Kai Petersen,et al.  Testing highly complex system of systems: An industrial case study , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[2]  Andrea Polini,et al.  An Extensible Framework for Online Testing of Choreographed Services , 2014, Computer.

[3]  H.-G. Gross,et al.  Testing Challenges of Maritime Safety and Security Systems-of-Systems , 2008, Testing: Academic & Industrial Conference - Practice and Research Techniques (taic part 2008).

[4]  Ricardo Valerdi,et al.  Test and evaluation of a SoS using a prescriptive and adaptive testing framework , 2010, 2010 5th International Conference on System of Systems Engineering.

[5]  Ricardo Pineda,et al.  Integration, Verification, Validation, Test, and Evaluation (IVVT&E) Framework for System of Systems (SoS) , 2013, Complex Adaptive Systems.

[6]  Doo-Hwan Bae,et al.  Mutation Analysis for System of Systems Policy Testing , 2017, 2017 IEEE/ACM Joint 5th International Workshop on Software Engineering for Systems-of-Systems and 11th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (JSOS).

[7]  Mark W. Maier.  Architecting Principles for Systems‐of‐Systems , 1996 .

[8]  Andrea Polini,et al.  SOA Test Governance: Enabling Service Integration Testing across Organization and Technology Borders , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[9]  Antonello Calabrò,et al.  When the Testing Gets Tough, the Tough Get ElasTest , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion).

[10]  Rogério de Lemos,et al.  Software Engineering for Self-Adaptive Systems: Research Challenges in the Provision of Assurances , 2013, Software Engineering for Self-Adaptive Systems.

[11]  Andrea Polini,et al.  Enhancing Service Federation Trustworthiness through Online Testing , 2012, Computer.

[12]  M. Janishidi.  System of Systems - Innovations for 21st Century , 2008, 2008 IEEE Region 10 and the Third international Conference on Industrial and Information Systems.

[13]  Mohammad Jamshidi,et al.  System of systems engineering : innovations for the 21st century , 2008 .

[14]  Eda Marchetti,et al.  Bringing white-box testing to Service Oriented Architectures through a Service Oriented Approach , 2011, J. Syst. Softw..

[15]  Jan Peleska,et al.  Systems of Systems Engineering , 2015 .

[16]  Flávio Oquendo,et al.  A mission-oriented approach for designing system-of-systems , 2015, 2015 10th System of Systems Engineering Conference (SoSE).

[17]  Antonia Bertolino,et al.  Applying Structural Testing to Services Using Testing Interfaces and Metadata , 2013, Int. J. Softw. Informatics.

[18]  David S. Rosenblum.  Adequate Testing of Component-Based Software , 1997 .

[19]  Stuart Harvey Rubin,et al.  Randomization for testing systems of systems , 2009, 2009 IEEE International Conference on Information Reuse & Integration.

[20]  Jo Ann Lane,et al.  Systems of systems test and evaluation challenges , 2010, 2010 5th International Conference on System of Systems Engineering.

[21]  Axel Legay,et al.  An Application of SMC to continuous validation of heterogeneous systems , 2017, EAI Endorsed Trans. Ind. Networks Intell. Syst..

[22]  Andrea Polini,et al.  Model-Based Generation of Testbeds for Web Services , 2008, TestCom/FATES.

[23]  Eda Marchetti,et al.  WS-TAXI: A WSDL-based Testing Tool for Web Services , 2009, 2009 International Conference on Software Testing Verification and Validation.

[24]  Chin-Yu Huang,et al.  An Adaptive Reliability Analysis Using Path Testing for Complex Component-Based Software Systems , 2011, IEEE Transactions on Reliability.

[25]  John M. Colombi,et al.  Interoperability Test and Evaluation: A System of Systems Field Study , 2008 .

[26]  Stuart Harvey Rubin,et al.  Randomization in Searching for Composites of Software Components , 2007, 2007 IEEE International Conference on Information Reuse and Integration.

[27]  Nenad Medvidovic,et al.  Unmanned and Autonomous Systems of Systems Test and Evaluation: Challenges and Opportunities , 2010 .

[28]  Ricardo Pineda,et al.  Basis Path Analysis for Testing Complex System of Systems , 2013, Complex Adaptive Systems.

[29]  Bill Wood,et al.  Introduction to the Mission Thread Workshop , 2013 .