Risk-based compatibility analysis in automotive systems engineering

Software is the new leading factor for innovation in the automotive industry. With the increase of software in road vehicles new business models, such as after-sale updates (i.e., Function-on-Demand) and Over-the-Air-Updates come into focus of manufacturers. When updating a road vehicle in the field, it is required to ensure functional safety. An update shall not influence existing functionality and break its safety. Hence, it must be compatible with the existing software. The compatibility of an update is ensured by testing. However, testing all variants of a highly configurable system, such as a modern car's software, is infeasible, due to the combinatorial explosion. To address this problem, in this paper, we propose a risk-based change-impact analysis to identify system variants relevant for retesting after an update. We combine existing concepts from product sampling, risk-based testing, and configuration prioritization and apply them to automotive architectures. For validating our concept, we use the Body Comfort System case study from the automotive industry. Our evaluation reveals that the concept backed by tool support may reduce testing effort by identifying and prioritizing incompatible variants wrt to a system update.

[1]  Sergio Segura,et al.  Variability testing in the wild: the Drupal case study , 2017, Software & Systems Modeling.

[2]  Hassan Gomaa,et al.  Designing Software Product Lines with UML 2.0: From Use Cases to Pattern-Based Software Architectures , 2006, 10th International Software Product Line Conference (SPLC'06).

[3]  Øystein Haugen,et al.  An algorithm for generating t-wise covering arrays from large feature models , 2012, SPLC '12.

[4]  Sandro Schulze,et al.  Risk-based integration testing of software product lines , 2017, VaMoS.

[5]  Sebastian Krieter,et al.  Product Sampling for Product Lines: The Scalability Challenge , 2019, SPLC.

[6]  Ståle Amland Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[7]  Remo Lachmann Black-Box Test Case Selection and Prioritization for Software Variants and Versions , 2017 .

[8]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[9]  Vasek Chvátal,et al.  A Greedy Heuristic for the Set-Covering Problem , 1979, Math. Oper. Res..

[10]  Uirá Kulesza,et al.  Safe evolution templates for software product lines , 2015, J. Syst. Softw..

[11]  Viviana Bono,et al.  Delta-Oriented Programming of Software Product Lines , 2010, SPLC.

[12]  Gunter Saake,et al.  Effective product-line testing using similarity-based product prioritization , 2016, Software & Systems Modeling.

[13]  Malte Lochau,et al.  Towards incremental test suite optimization for software product lines , 2014, FOSD '14.

[14]  Michael Nieke,et al.  Anomaly analyses for feature-model evolution , 2018, GPCE.

[15]  Sandro Schulze,et al.  Delta-oriented test case prioritization for integration testing of software product lines , 2015, SPLC.

[16]  Ina Schaefer,et al.  Variability Modelling for Model-Driven Development of Software Product Lines , 2010, VaMoS.

[17]  Goetz Botterweck,et al.  Software diversity: state of the art and perspectives , 2012, International Journal on Software Tools for Technology Transfer.

[18]  Iago Abal,et al.  42 variability bugs in the linux kernel: a qualitative analysis , 2014, ASE.

[19]  Malte Lochau,et al.  A Comprehensive Description of a Model-based, Continuous Development Process for AUTOSAR Systems with Integrated Quality Assurance , 2009 .

[20]  Jan Bosch,et al.  Risk based testing for software product line engineering , 2014, SPLC.

[21]  AmlandStåle Risk-based testing , 2000 .

[22]  Mustafa Al-Hajjaji,et al.  Similarity-driven prioritization and sampling for product-line testing , 2017 .

[23]  Manfred Broy,et al.  Automotive software engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[24]  Yan Li,et al.  Approaches for the combined use of risk analysis and testing: a systematic literature review , 2014, International Journal on Software Tools for Technology Transfer.

[25]  Hassan Gomaa Designing Software Product Lines with UML 2.0: From Use Cases to Pattern-Based Software Architectures , 2006, ICSR.

[26]  Dragan Gasevic,et al.  Goal-Oriented Test Case Selection and Prioritization for Product Line Feature Models , 2011, 2011 Eighth International Conference on Information Technology: New Generations.

[27]  Sven Apel,et al.  A Comparison of 10 Sampling Algorithms for Configurable Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[28]  Sophia Nahrendorf,et al.  Applying Higher-Order Delta Modeling for the Evolution of Delta-Oriented Software Product Lines , 2018 .

[29]  Malte Lochau,et al.  Pairwise feature-interaction testing for SPLs: potentials and limitations , 2011, SPLC '11.

[30]  Myra B. Cohen,et al.  Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach , 2008, IEEE Transactions on Software Engineering.

[31]  Jacques Klein,et al.  Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines , 2014, IEEE Transactions on Software Engineering.

[32]  Udo Kelter,et al.  Reasoning about product-line evolution using complex feature model differences , 2016, Automated Software Engineering.

[33]  Gunter Saake,et al.  Feature-Oriented Software Product Lines , 2013, Springer Berlin Heidelberg.

[34]  Krzysztof Czarnecki,et al.  Generative programming - methods, tools and applications , 2000 .

[35]  Michael Nieke,et al.  Back to the future: avoiding paradoxes in feature-model evolution , 2018, SPLC.

[36]  Michal Antkiewicz,et al.  Mapping features to models: a template approach based on superimposed variants , 2005, GPCE'05.

[37]  Sungwon Kang,et al.  A survey on software product line testing , 2012, SPLC '12.

[38]  Mustafa Al-Hajjaji,et al.  A classification of product sampling for software product lines , 2018, SPLC.

[39]  Thomas Thüm,et al.  Reasoning about edits to feature models , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[40]  Malte Lochau,et al.  Delta-oriented Software Product Line Test Models - The Body Comfort System Case Study , 2013 .

[41]  Pierre-Yves Schobbens,et al.  Towards statistical prioritization for software product lines testing , 2013, VaMoS.

[42]  Ina Schieferdecker,et al.  A taxonomy of risk-based testing , 2014, International Journal on Software Tools for Technology Transfer.

[43]  Alexander Egyed,et al.  A first systematic mapping study on combinatorial interaction testing for software product lines , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[44]  Ina Schaefer,et al.  Higher-order delta modeling for software product line evolution , 2016, FOSD.

[45]  Hans-Leo Ross Functional Safety for Road Vehicles: New Challenges and Solutions for E-mobility and Automated Driving , 2016 .

[46]  Sebastian Krieter,et al.  Feature-Model Interfaces: The Highway to Compositional Analyses of Highly-Configurable Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).