State-based software testing

Software testing is an important and a costly phase of the software development lifecycle. In this research, an approach is developed that improves the effectiveness of software testing based on partitioning the input domain. Specifically, it is demonstrated how partitioning can decrease the number of tests needed to find defects in black-box software testing. The improvement is based on partitioning the input domain according to how different user-level functions of software read from and write to internal state variables. Software functions are represented by state traces that are derived from the software's state box design. The use of such state-based partitioning approach is demonstrated by applying it in the statistical reliability certification phase of Cleanroom software engineering. Several parameters that influence the probability of finding defects are investigated with a computer simulation: the size of a test case, the type of a defect, the number of partitions, and the ratio of reading and writing accesses to state variables. Based on these parameters, partitions can be established that increase the probability of finding defects. The effectiveness of this approach is demonstrated with three case studies that compare the number of tests needed to find seeded defects in unpartitioned and partitioned testing scenarios. The case studies resulted in a 38%, 75%, and 37% reduction in the number of tests needed to find defects during partitioned testing. Additional applications of this approach that are investigated include automatic test-case generation and improving defect localization.