Scalable data-flow testing

ARAUJO, Roberto Paulo Andrioli de. Scalable data-flow testing. 2014. 100 p. Dissertation (Master of Science) – School of Arts, Sciences and Humanities, University of Sao Paulo, Sao Paulo, 2014. Data-flow (DF) testing was introduced more than thirty years ago aiming at verifying a program by extensively exploring its structure. It requires tests that traverse paths in which the assignment of a value to a variable (a definition) and its subsequent reference (a use) is verified. This relationship is called definition-use association (dua). While control-flow (CF) testing tools have being able to tackle systems composed of large and long running programs, DF testing tools have failed to do so. This situation is in part due to the costs associated with tracking duas at run-time. Recently, an algorithm, called Bitwise Algorithm (BA), which uses bit vectors and bitwise operations for tracking intra-procedural duas at run-time, was proposed. This research presents the implementation of BA for programs compiled into Java bytecodes. Previous DF approaches were able to deal with small to medium size programs with high penalties in terms of execution and memory. Our experimental results show that by using BA we are able to tackle large systems with more than 250 KLOCs and 300K required duas. Furthermore, for several programs the execution penalty was comparable with that imposed by a popular CF testing tool.

[1]  Marcos Lordello Chaim,et al.  An efficient bitwise algorithm for intra-procedural data-flow testing coverage , 2013, Inf. Process. Lett..

[2]  Phyllis G. Frankl,et al.  Further empirical studies of test effectiveness , 1998, SIGSOFT '98/FSE-6.

[3]  Hira Agrawal,et al.  Efficient coverage testing using global dominator graphs , 1999, ACM SIGSOFT Softw. Eng. Notes.

[4]  Etsuya Shibayama,et al.  Security Sensitive Data Flow Coverage Criterion for Automatic Security Testing of Web Applications , 2011, ESSoS.

[5]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[6]  W. Eric Wong,et al.  Coverage testing of Java programs and components , 2005, Sci. Comput. Program..

[7]  Jeffrey K. Hollingsworth,et al.  Efficient online computation of statement coverage , 2005, J. Syst. Softw..

[8]  Mary Lou Soffa,et al.  Demand-driven structural testing with dynamic instrumentation , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[9]  Jun Yan,et al.  An efficient method to generate feasible paths for basis path testing , 2008, Inf. Process. Lett..

[10]  Sebastian Fischmeister,et al.  Sampling-based program execution monitoring , 2010, LCTES '10.

[11]  Dror G. Feitelson,et al.  Development and Deployment at Facebook , 2013, IEEE Internet Computing.

[12]  Mary Lou Soffa,et al.  Exploiting hardware advances for software testing and debugging: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[13]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[14]  Raúl A. Santelices,et al.  Efficiently monitoring data-flow test coverage , 2007, ASE.

[15]  Ilona Bluemke,et al.  Dataflow Approach to Testing Java Programs , 2009, 2009 Fourth International Conference on Dependability of Computer Systems.

[16]  Alessandro Orso,et al.  InsECTJ: a generic instrumentation framework for collecting dynamic information within Eclipse , 2005, eclipse '05.

[17]  Qian Yang,et al.  A survey of coverage based testing tools , 2006, AST '06.

[18]  Elaine J. Weyuker,et al.  Selecting Software Test Data Using Data Flow Information , 1985, IEEE Transactions on Software Engineering.

[19]  Marcos Lordello Chaim,et al.  Data-Flow Testing in the Large , 2014, ICST.

[20]  Elaine J. Weyuker,et al.  Data flow analysis techniques for test data selection , 2015, ICSE '82.

[21]  J. R. Horgan,et al.  A data flow coverage testing tool for C , 1992, [1992] Proceedings of the Second Symposium on Assessment of Quality Software Development Tools.

[22]  Raúl A. Santelices,et al.  Lightweight fault-localization using multiple coverage types , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[23]  Elaine J. Weyuker,et al.  Data flow-based test adequacy analysis for languages with pointers , 1991, TAV4.

[24]  Antonia Bertolino,et al.  Using Spanning Sets for Coverage Testing , 2003, IEEE Trans. Software Eng..

[25]  James H. Andrews,et al.  Comparing Multi-Point Stride Coverage and dataflow coverage , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[26]  Laurie Hendren,et al.  Soot---a java optimization framework , 1999 .

[27]  Mary Lou Soffa,et al.  Jazz: A Tool for Demand-Driven Structural Testing , 2005, CC.

[28]  Gilad Bracha,et al.  The Java Virtual Machine Specification, Java SE 8 Edition , 2013 .

[29]  James R. Larus,et al.  Efficient path profiling , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.