Adaptive Testing Based on Moment Estimation

Adaptive testing (AT) is a software testing approach that uses a feedback mechanism to enhance test effectiveness. Its testing strategy can be adjusted online by using the testing data collected during the software testing process. However, it requires complex parameter estimation which results in excessive computational overhead that may hinder the applicability of AT. In this paper, we propose an approach called AT based on moment estimation (AT-ME) to address this problem. The proposed approach uses moment estimation to serve as the algorithm of parameter estimation, which reduces the complexity of AT-ME. In addition, a dynamic length for testing action is set to limit the number of decisions without influencing the test effectiveness. The proposed approach has been validated on the Siemens test suite, which includes seven real programs. The experiments show that AT-ME can reduce the computational overhead of AT without compromising overall testing efficiency. Results demonstrate that AT-ME is a feasible and effective AT strategy.

[1]  Kai-Yuan Cai,et al.  Optimal and adaptive testing for software reliability assessment , 2004, Inf. Softw. Technol..

[2]  Lingxia Zhang,et al.  Comparison of random and partition testing considering the test profile , 2003 .

[3]  Bojan Cukic,et al.  Caveats , 2020, The African Continental Free Trade Area: Economic and Distributional Effects.

[4]  Kai-Yuan Cai,et al.  Adaptive and Random Partition Software Testing , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Kai-Yuan Cai,et al.  A Parallel Implementation Strategy of Adaptive Testing , 2010, 2010 IEEE 34th Annual Computer Software and Applications Conference Workshops.

[6]  Kao-Shing Hwang,et al.  Reinforcement learning to adaptive control of nonlinear systems , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Jiang Tong-min,et al.  On modeling approach for embedded real-time software simulation testing , 2009 .

[8]  Hareton K. N. Leung,et al.  A Case Study of Adaptive Combinatorial Testing , 2013, 2013 IEEE 37th Annual Computer Software and Applications Conference Workshops.

[9]  Kai-Yuan Cai,et al.  A case study of the recursive least squares estimation approach to adaptive testing for software components , 2005, Fifth International Conference on Quality Software (QSIC'05).

[10]  Bojan Cukic,et al.  Comparing Partition and Random Testing via Majorization and Schur Functions , 2003, IEEE Trans. Software Eng..

[11]  Kai-Yuan Cai,et al.  Optimal software testing in the setting of controlled Markov chains , 2005, Eur. J. Oper. Res..

[12]  Kai-Yuan Cai,et al.  Enhancing software reliability estimates using modified adaptive testing , 2013, Inf. Softw. Technol..

[13]  Kai-Yuan Cai,et al.  Adaptive software testing with fixed-memory feedback , 2007, J. Syst. Softw..

[14]  Kai-Yuan Cai,et al.  On the Computational Complexity of Parameter Estimation in Adaptive Testing Strategies , 2009, 2009 15th IEEE Pacific Rim International Symposium on Dependable Computing.

[15]  Kai-Yuan Cai,et al.  An experimental study of adaptive testing for software reliability assessment , 2008, J. Syst. Softw..

[16]  F. Bastani,et al.  Weighted Proportional Sampling : AGeneralization for Sampling Strategies in Software Testing , 2006, 2006 3rd International Conference on Electrical and Electronics Engineering.

[17]  Kai-Yuan Cai,et al.  On the Online Parameter Estimation Problem in Adaptive Software Testing , 2008, Int. J. Softw. Eng. Knowl. Eng..

[18]  Anneliese Amschler Andrews,et al.  Fast antirandom (FAR) test generation , 1998, Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No.98EX231).

[19]  Kai-Yuan Cai,et al.  Adaptive Software Testing in the Context of an Improved Controlled Markov Chain Model , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[20]  HAI HU,et al.  An Improved Approach to Adaptive Testing , 2009, Int. J. Softw. Eng. Knowl. Eng..

[21]  I. K. Mak,et al.  Adaptive Random Testing , 2004, ASIAN.

[22]  Hareton K. N. Leung,et al.  Adaptive Combinatorial Testing , 2013, 2013 13th International Conference on Quality Software.

[23]  Tsong Yueh Chen,et al.  On the Relationship Between Partition and Random Testing , 1994, IEEE Trans. Software Eng..

[24]  Yashwant K. Malaiya,et al.  Antirandom testing: getting the most out of black-box testing , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[25]  Kai-Yuan Cai,et al.  On the Asymptotic Behavior of Adaptive Testing Strategy for Software Reliability Assessment , 2014, IEEE Transactions on Software Engineering.

[26]  Kai-Yuan Cai,et al.  How to test software for optimal software reliability assessment , 2003, Third International Conference on Quality Software, 2003. Proceedings..

[27]  Abraham Kandel,et al.  A Comparative Study of Artificial Neural Networks and Info-Fuzzy Networks as Automated Oracles in Software Testing , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[28]  Kai-Yuan Cai,et al.  Optimal and adaptive testing with cost constraints , 2006, AST '06.

[29]  Kai-Yuan Cai,et al.  Optimal software testing and adaptive software testing in the context of software cybernetics , 2002, Inf. Softw. Technol..

[30]  Tsong Yueh Chen,et al.  On Adaptive Random Testing Through Iterative Partitioning , 2006, J. Inf. Sci. Eng..

[31]  Kai-Yuan Cai,et al.  Towards research on software cybernetics , 2002, 7th IEEE International Symposium on High Assurance Systems Engineering, 2002. Proceedings..