Optimal stopping of multi-project software testing in the context of software cybernetics

Software cybernetics explores the interplay between control theory/engineering and software theory/engineering. The controlled Markov chains (CMC) approach to software testing follows the idea of software cybernetics and treats software testing as a control problem. The software under test serves as a controlled object and the software testing strategy serves as the corresponding controller. The software under test and the software testing strategy make up a closed-loop feedback control system, and the theory of controlled Markov chains can be used to design and optimize software testing strategies in accordance with testing/reliability goals given a priori. In this paper we apply the CMC approach to the optimal stopping problem of multi-project software testing. The problem under consideration assumes that a single stopping action can stop testing of all the software systems under test simultaneously. The theoretical results presented in this paper describe how to test multiple software systems and when to stop testing in an optimal manner. An illustrative example is used to explain the theoretical results. The study of this paper further justifies the effectiveness of the CMC approach to software testing in particular and the idea of software cybernetics in general.

[1]  Elaine J. Weyuker,et al.  Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study , 2000, IEEE Trans. Software Eng..

[2]  Boris Beizer,et al.  Rebuttal of the book review by R. Whitty of ‘Software Testing Techniques’ (2nd edn) , 1994, Softw. Test. Verification Reliab..

[3]  Kai-Yuan Cai,et al.  Optimal software testing and adaptive software testing in the context of software cybernetics , 2002, Inf. Softw. Technol..

[4]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[5]  Kai-Yuan Cai,et al.  Towards a conceptual framework of software run reliability modeling , 2000, Inf. Sci..

[6]  Tsong Yueh Chen,et al.  On the Expected Number of Failures Detected by Subdomain Testing and Random Testing , 1996, IEEE Trans. Software Eng..

[7]  Elaine J. Weyuker,et al.  A Formal Analysis of the Fault-Detecting Ability of Testing Methods , 1993, IEEE Trans. Software Eng..

[8]  Elaine J. Weyuker,et al.  The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software , 1995, IEEE Trans. Software Eng..

[9]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[10]  Walter J. Gutjahr,et al.  Partition Testing vs. Random Testing: The Influence of Uncertainty , 1999, IEEE Trans. Software Eng..

[11]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[12]  James A. Whittaker,et al.  What is software testing? And why is it so hard? , 2000 .

[13]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[14]  Kai-Yuan Cai Optimal test profile in the context of software cybernetics , 2001, Proceedings Second Asia-Pacific Conference on Quality Software.