Examining the co-evolution relationship between simulink models and their test cases

This paper presents an industrial case study that explores the co-evolution relationship between Matlab Simulink Models and their associated test suites. Through an analysis of differences between releases of both the models and their tests, we are able to determine what the relation between the model evolution and test evolution is, or if one exists at all. Using this comparison methodology, we present empirical results from a production system of 64 Matlab Simulink Models evolving over 9 releases. In our work we show that in this system there is a strong co-evolution relationship (a correlation value of r = 0.9, p < 0.01) between the models and tests, and we examine the cases where the relationship does not exist. We also pose, and answer, three specific research questions about the practices of development and testing over time for the system under study.

[1]  Alessandro Orso,et al.  Understanding myths and realities of test-suite evolution , 2012, SIGSOFT FSE.

[2]  Cristian Cadar,et al.  Covrig: a framework for the analysis of code, test, and coverage evolution in real software , 2014, ISSTA 2014.

[3]  M.M. Lehman,et al.  Programs, life cycles, and laws of software evolution , 1980, Proceedings of the IEEE.

[4]  James Everett Young,et al.  ChronoTwigger: A Visual Analytics Tool for Understanding Source and Test Co-evolution , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[5]  Richard F. Paige,et al.  Evolving models in Model-Driven Engineering: State-of-the-art and future challenges , 2016, J. Syst. Softw..

[6]  Lionel C. Briand,et al.  Automated Test Suite Generation for Time-Continuous Simulink Models , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[7]  Mauro Pezzè,et al.  Automatically repairing test cases for evolving method declarations , 2010, 2010 IEEE International Conference on Software Maintenance.

[8]  Andy Zaidman,et al.  Studying Fine-Grained Co-evolution Patterns of Production and Test Code , 2014, 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation.

[9]  Ruth Breu,et al.  A Generic Platform for Model-Based Regression Testing , 2012, ISoLA.

[10]  Andy Zaidman,et al.  Using association rules to study the co-evolution of production & test code , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[11]  Xin Yao,et al.  A novel co-evolutionary approach to automatic software bug fixing , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[12]  Eric James Rapos.  Co-Evolution of Model-Based Tests for Industrial Automotive Software , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[13]  Arie van Deursen,et al.  Mining Software Repositories to Study Co-Evolution of Production & Test Code , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[14]  Rajesh Subramanyan,et al.  A survey on model-based testing approaches: a systematic review , 2007, WEASELTech '07.

[15]  Arie van Deursen,et al.  Studying the co-evolution of production and test code in open source and industrial developer test processes through repository mining , 2008, Empirical Software Engineering.