Localizing Multiple Faults in Simulink Models

As Simulink is a widely used language in the embedded industry, there is a growing need to support debugging activities for Simulink models. In this work, we propose an approach to localize multiple faults in Simulink models. Our approach builds on statistical debugging and is iterative. At each iteration, we identify and resolve one fault and re-test models to focus on localizing faults that might have been masked before. We use decision trees to cluster together failures that satisfy similar (logical) conditions on model blocks or inputs. We then present two alternative selection criteria to choose a cluster that is more likely to yield the best fault localization results among the clusters produced by our decision trees. Engineers are expected to inspect the ranked list obtained from the selected cluster to identify faults. We evaluate our approach on 240 multi-fault models obtained from three different industrial subjects. We compare our approach with two baselines: (1) Statistical debugging without clustering, and (2) State-of-the-art clustering-based statistical debugging. Our results show that our approach significantly reduces the number of blocks that engineers need to inspect in order to localize all faults, when compared with the two baselines. Furthermore, with our approach, there is less performance degradation than in the baselines when increasing the number of faults in the underlying models.

[1]  Pieter J. Mosterman,et al.  Model-Based Testing for Embedded Systems , 2011, Computational Analysis, Synthesis, & Design Dynamic Systems.

[2]  Xiangyu Zhang,et al.  Precise dynamic slicing algorithms , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[3]  David Lo,et al.  Fusion fault localizers , 2014, ASE.

[4]  Olivier Ridoux,et al.  Multiple Fault Localization with Data Mining , 2011, SEKE.

[5]  Alex Groce,et al.  Understanding Counterexamples with explain , 2004, CAV.

[6]  Paweł Skruch,et al.  Model-Based Testing in Embedded Automotive Systems , 2011, Model-Based Testing for Embedded Systems.

[7]  Sabine Glesner,et al.  Slicing MATLAB Simulink models , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[8]  Michael I. Jordan,et al.  Statistical debugging: simultaneous identification of multiple bugs , 2006, ICML '06.

[9]  Andreas Zeller,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[10]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[11]  L. Lundberg,et al.  Advances in Computer Science - ASIAN 2004. Higher-Level Decision Making , 2005 .

[12]  Mary Jean Harrold,et al.  Debugging in Parallel , 2007, ISSTA '07.

[13]  Andreas Zeller,et al.  Simplifying failure-inducing input , 2000, ISSTA '00.

[14]  James A. Jones,et al.  Visualization of test information to assist fault localization , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[15]  Alessandro Orso,et al.  Are automated debugging techniques actually helping programmers? , 2011, ISSTA '11.

[16]  Andreas Zeller,et al.  Finding Failure Causes through Automated Testing , 2000, AADEBUG.

[17]  Steven P. Reiss,et al.  Fault localization with nearest neighbor queries , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[18]  Yuriy Brun,et al.  Finding latent code errors via machine learning over program executions , 2004, Proceedings. 26th International Conference on Software Engineering.

[19]  Peter Zoeteweij,et al.  Spectrum-Based Multiple Fault Localization , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[20]  Alessandro Orso,et al.  GAMMATELLA: visualization of program-execution data for deployed software , 2004, Proceedings. 26th International Conference on Software Engineering.

[21]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[22]  T. Y. Chen,et al.  Adaptive Random Testing , 2004, ASIAN.

[23]  Friedrich Steimann,et al.  Improving Coverage-Based Localization of Multiple Faults Using Algorithms from Integer Linear Programming , 2012, 2012 IEEE 23rd International Symposium on Software Reliability Engineering.

[24]  Mayur Naik,et al.  From symptom to cause: localizing errors in counterexample traces , 2003, POPL '03.

[25]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[26]  徐锋,et al.  基于 Stateflow 的电动汽车再生制动控制策略 , 2014 .

[27]  Peter Zoeteweij,et al.  Localizing Software Faults Simultaneously , 2009, 2009 Ninth International Conference on Quality Software.

[28]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[29]  Lionel C. Briand,et al.  Simulink fault localization: an iterative statistical debugging approach , 2016, Softw. Test. Verification Reliab..

[30]  Chao Liu,et al.  SOBER: statistical model-based bug localization , 2005, ESEC/FSE-13.

[31]  Jochen Quante,et al.  Reengineering embedded automotive software , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[32]  Lionel C. Briand,et al.  Using Machine Learning to Support Debugging with Tarantula , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[33]  Ben Liblit,et al.  Adaptive bug isolation , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[34]  Alexander Egyed,et al.  Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering , 2007, ASE 2007.

[35]  Chao Liu,et al.  Failure proximity: a fault localization-based approach , 2006, SIGSOFT '06/FSE-14.

[36]  Bin Wang,et al.  Automated support for classifying software failure reports , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[37]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[38]  Zhendong Su,et al.  Context-aware statistical debugging: from bug predictors to faulty control flow paths , 2007, ASE.

[39]  Alessandro Orso,et al.  Isolating failure causes through test case generation , 2012, ISSTA 2012.

[40]  Rui Abreu,et al.  Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators , 2013, ISSTA.

[41]  R. Grissom,et al.  Effect sizes for research: A broad practical approach. , 2005 .

[42]  Adepu Sridhar,et al.  Slicing MATLAB Simulink/Stateflow Models , 2013, ICACNI.