Trace Reduction and Pattern Analysis to Assist Debugging in Model-Based Testing

Model-based testing (MBT) is a technique for generating test cases from test models. One of the benefits of MBT is the ability to have a computer generate and execute extensive test sets from the test models, achieving high coverage. However, when such large test sets are automatically generated and executed, the resulting failure traces can be very large and difficult to debug for root cause analysis. In this paper, we present a technique for minimizing the length of a failure trace, creating variants of it, and for pattern mining the trace variants to assist in root cause analysis. We demonstrate the technique on a model of a GSM SIM card.

[1]  David Lo,et al.  Scenario-based and value-based specification mining: better together , 2010, Automated Software Engineering.

[2]  Jurriaan Hage,et al.  Guided Algebraic Specification Mining for Failure Simplification , 2013, ICTSS.

[3]  Kim G. Larsen,et al.  T-UPPAAL: online model-based testing of real-time systems , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[4]  Peter Zoeteweij,et al.  Simultaneous debugging of software faults , 2011, J. Syst. Softw..

[5]  Teemu Kanstrén,et al.  Using Built-In Domain-Specific Modeling Support to Guide Model-Based Test Generation , 2012, MBT.

[6]  Jiawei Han,et al.  Mining Software Specifications: Methodologies and Applications , 2011 .

[7]  Alessandro Orso,et al.  Isolating failure causes through test case generation , 2012, ISSTA 2012.

[8]  Antti Huima,et al.  Implementing Conformiq Qtronic , 2007, TestCom/FATES.

[9]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[10]  Cesare Pautasso,et al.  Specification patterns from research to industry: A case study in service-based applications , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[11]  Andreas Zeller,et al.  Efficient unit test case minimization , 2007, ASE '07.

[12]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[13]  George S. Avrunin,et al.  Patterns in property specifications for finite-state verification , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[14]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[15]  Howard Barringer,et al.  A pattern-based approach to parametric specification mining , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[16]  Frank Tip,et al.  Fault Localization for Dynamic Web Applications , 2012, IEEE Transactions on Software Engineering.

[17]  Xiangyu Zhang,et al.  Comparative causality: Explaining the differences between executions , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[18]  Margus Veanes,et al.  Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer , 2008, Formal Methods and Testing.

[19]  A. Zeller Isolating cause-effect chains from computer programs , 2002, SIGSOFT '02/FSE-10.

[20]  Wolfgang Grieskamp,et al.  Model‐based quality assurance of protocol documentation: tools and methodology , 2011, Softw. Test. Verification Reliab..

[21]  Marko Kääramees,et al.  Constraint-Based Heuristic On-line Test Generation from Non-deterministic I/O EFSMs , 2012, MBT.