Complete Shadow Symbolic Execution with Java PathFinder

Regression testing ensures the correctness of the software during its evolution, with special attention on the absence of unintended side-e ects that might be introduced by changes. However, the manual creation of regression test cases, which expose divergent behavior, needs a lot of e ort. A solution is the idea of shadow symbolic execution, which takes a uni ed version of the old and the new programs and performs symbolic execution guided by concrete values to explore the changed behavior. In this work, we adapt the idea of shadow symbolic execution (SSE) and combine complete/standard symbolic execution with the idea of four-way forking to expose diverging behavior. There- fore, our approach attempts to comprehensively test the new be- haviors introduced by a change. We implemented our approach in the tool ShadowJPF+, which performs complete shadow sym- bolic execution on Java bytecode. It is an extension of the tool ShadowJPF, which is based on Symbolic PathFinder. We applied our tool on 79 examples, for which it was able to reveal more di- verging behaviors than common shadow symbolic execution. Ad- ditionally, the approach has been applied on a real-world patch for the Joda-Time library, for which it successfully generated test cases that expose a regression error.

[1]  Gregg Rothermel,et al.  An empirical study of regression test selection techniques , 1998, Proceedings of the 20th International Conference on Software Engineering.

[2]  Cristian Cadar,et al.  KATCH: high-coverage testing of software patches , 2013, ESEC/FSE 2013.

[3]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[4]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[5]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[6]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[7]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[8]  René Just,et al.  MAJOR: An efficient and extensible tool for mutation analysis in a Java compiler , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[9]  Corina S. Pasareanu,et al.  Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis , 2013, Automated Software Engineering.

[10]  Michael Hicks,et al.  Directed Symbolic Execution , 2011, SAS.

[11]  Zhendong Su,et al.  Has the bug really been fixed? , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[12]  Alessandro Orso,et al.  Regression test selection for Java software , 2001, OOPSLA '01.

[13]  Timo Kehrer,et al.  Shadow Symbolic Execution with Java PathFinder , 2018, SOEN.

[14]  Ding Yuan,et al.  How do fixes become bugs? , 2011, ESEC/FSE '11.

[15]  Cristian Cadar,et al.  Shadow of a Doubt: Testing for Divergences between Software Versions , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[16]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.