Variable Strength Combinatorial Testing for Deep Neural Networks

In deep neural networks (DNNs), each neuron in the post-layer receives the influence of all the neurons in the pre-layer. As we known, different connections in a DNN model have different weights. It means that, different combinations of pre-layer neurons have different effects on the post-layer neurons. Therefore, the variable strength combinatorial testing can reflect the effect of combination interaction of neurons in the pre-layer on the neurons in the post-layer. In this paper, we propose to adopt variable strength combinatorial testing technique on DNNs testing. In order to modeling the effect of combinatorial interaction of pre-layer neurons on the post-layer neurons, we propose three methods to construct variable strength combinatorial interaction relationship for DNNs. The experimental results show that, 1) variable strength combinatorial coverage criteria are discriminating to measure the adequacy of DNNs testing, and 2) there is correlation between variable strength combinatorial coverage and adversarial detection.