Enhancing Performance of Random Testing through Markov Chain Monte Carlo Methods

In this paper, we propose a probabilistic approach to finding failure-causing inputs based on Bayesian estimation. According to our probabilistic insights of software testing, the test case generation algorithms are developed by Markov chain Monte Carlo (MCMC) methods. Dissimilar to existing random testing schemes such as adaptive random testing, our approach can also utilize the prior knowledge on software testing. In experiments, we compare effectiveness of our MCMC-based random testing with both ordinary random testing and adaptive random testing in real program sources. These results indicate the possibility that MCMC-based random testing can drastically improve the effectiveness of software testing.

[1]  Tadashi Dohi,et al.  Markov Chain Monte Carlo Random Testing , 2010, AST/UCMA/ISA/ACN.

[2]  Bertrand Meyer,et al.  ARTOO: adaptive random testing for object-oriented software , 2008, ICSE.

[3]  T. Y. Chen,et al.  Adaptive Random Testing , 2004, ASIAN.

[4]  Tsong Yueh Chen,et al.  On the Expected Number of Failures Detected by Subdomain Testing and Random Testing , 1996, IEEE Trans. Software Eng..

[5]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[6]  Tsong Yueh Chen,et al.  Proportional sampling strategy: a compendium and some insights , 2001, J. Syst. Softw..

[7]  Dave Towey,et al.  Forgetting Test Cases , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[8]  VISHWANI D. AGRAWAL When to Use Random Testing , 1978, IEEE Transactions on Computers.

[9]  李 鎔範,et al.  数値計算のためのGNU Scientific Libraryの紹介(教育講座) , 2012 .

[10]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[11]  Sheldon M. Ross,et al.  Introduction to Probability Models (4th ed.). , 1990 .

[12]  Tsong Yueh Chen,et al.  Proportional sampling strategy: guidelines for software testing practitioners , 1996, Inf. Softw. Technol..

[13]  Dave Towey,et al.  Normalized Restricted Random Testing , 2003, Ada-Europe.

[14]  Huai Liu,et al.  Application of a Failure Driven Test Profile in Random Testing , 2009, IEEE Trans. Reliab..

[15]  Tsong Yueh Chen,et al.  Mirror adaptive random testing , 2004, Inf. Softw. Technol..

[16]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[17]  Sheldon M. Ross Introduction to Probability Models. , 1995 .

[18]  Sheldon M. Ross,et al.  Introduction to Probability Models, Eighth Edition , 1972 .

[19]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[20]  Tsong Yueh Chen,et al.  An upper bound on software testing effectiveness , 2008, TSEM.

[21]  Zongyuan Yang,et al.  An Innovative Approach to Tackling the Boundary Effect in Adaptive Random Testing , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).