Exploiting the Largest Available Zone: A Proactive Approach to Adaptive Random Testing by Exclusion

Adaptive random testing (ART) has been proposed to enhance the effectiveness of random testing (RT) through more even spreading of the test cases. In particular, restricted random testing (RRT) is an ART algorithm based on the intuition of skipping all the candidate test cases that are within the neighborhoods (or zones) of previously executed test cases. RRT has higher effectiveness than RT in terms of failure detection but incurs a higher time cost. In this paper, we aim to further reduce the time costs for RRT and improve the effectiveness for RT and ART methods. We propose a proactive technique known as “RRT by largest available zone” (RRT-LAZ). Like RRT, RRT-LAZ first defines an exclusion zone around every executed test case in order to determine the available zones. Unlike the original RRT, RRT-LAZ then compares all the available zones to proactively pick the largest one, from which the next test case is randomly generated. Both simulation analyses and empirical studies have been employed to investigate the efficiency and effectiveness of RRT-LAZ in relation to RT and related ART algorithms. The results show that RRT-LAZ has significantly lower time costs than RRT. Furthermore, RRT-LAZ is more effective than RT and related ART methods for block failure patterns in low-dimensional input spaces. In general, since RRT-LAZ employs a proactive technique instead of a passive one in generating next cases, it is much more cost-effective than RRT. RRT-LAZ is also more cost-effective than RT and other ART methods that we have studied.

[1]  Dave Towey,et al.  Normalized Restricted Random Testing , 2003, Ada-Europe.

[2]  Huai Liu,et al.  A Cost-Effective Random Testing Method for Programs with Non-Numeric Inputs , 2016, IEEE Transactions on Computers.

[3]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[4]  Dave Towey,et al.  Restricted Random Testing: Adaptive Random Testing by Exclusion , 2006, Int. J. Softw. Eng. Knowl. Eng..

[5]  Walter J. Gutjahr,et al.  Partition Testing vs. Random Testing: The Influence of Uncertainty , 1999, IEEE Trans. Software Eng..

[6]  Tsong Yueh Chen,et al.  Adaptive Random Testing: The ART of test case diversity , 2010, J. Syst. Softw..

[7]  Tsong Yueh Chen,et al.  An upper bound on software testing effectiveness , 2008, TSEM.

[8]  James Miller,et al.  A Novel Evolutionary Approach for Adaptive Random Testing , 2009, IEEE Transactions on Reliability.

[9]  Jinfu Chen,et al.  A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing , 2017, IEEE Transactions on Reliability.

[10]  Tsong Yueh Chen,et al.  Adaptive random testing through dynamic partitioning , 2004, Fourth International Conference onQuality Software, 2004. QSIC 2004. Proceedings..

[11]  Peter G. Bishop,et al.  The variation of software survival time for different operational input profiles (or why you can wait a long time for a big bug to fail) , 1993, FTCS-23 The Twenty-Third International Symposium on Fault-Tolerant Computing.

[12]  Tsong Yueh Chen,et al.  Mirror adaptive random testing , 2004, Inf. Softw. Technol..

[13]  Jianjun Zhao,et al.  A Divergence-Oriented Approach to Adaptive Random Testing of Java Programs , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[14]  Chengying Mao,et al.  Adaptive Random Testing Based on Two-Point Partitioning , 2012, Informatica.

[15]  Lee J. White,et al.  A Domain Strategy for Computer Program Testing , 1980, IEEE Transactions on Software Engineering.

[16]  Huai Liu,et al.  Adaptive Random Testing by Exclusion through Test Profile , 2010, 2010 10th International Conference on Quality Software.

[17]  Tsong Yueh Chen,et al.  Proportional sampling strategy: guidelines for software testing practitioners , 1996, Inf. Softw. Technol..

[18]  Huai Liu,et al.  Application of a Failure Driven Test Profile in Random Testing , 2009, IEEE Trans. Reliab..

[19]  Will Tribbey,et al.  Numerical Recipes: The Art of Scientific Computing (3rd Edition) is written by William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery, and published by Cambridge University Press, © 2007, hardback, ISBN 978-0-521-88068-8, 1235 pp. , 1987, SOEN.

[20]  Bertrand Meyer,et al.  ARTOO , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[21]  Dave Towey,et al.  Restricted Random Testing , 2002, ECSQ.

[22]  Jinfu Chen,et al.  A cost-effective adaptive random testing approach by dynamic restriction , 2018, IET Softw..

[23]  Gregory Tassey,et al.  Prepared for what , 2007 .

[24]  Huai Liu,et al.  On Test Case Distributions of Adaptive Random Testing , 2007, SEKE.

[25]  I. K. Mak,et al.  Adaptive Random Testing , 2004, ASIAN.

[26]  Arthur R. Butz,et al.  Alternative Algorithm for Hilbert's Space-Filling Curve , 1971, IEEE Transactions on Computers.

[27]  Dave Towey,et al.  Forgetting Test Cases , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[28]  James Miller,et al.  Centroidal Voronoi Tessellations- A New Approach to Random Testing , 2013, IEEE Transactions on Software Engineering.

[29]  Johannes Mayer,et al.  An empirical analysis and comparison of random testing techniques , 2006, ISESE '06.