MofN rule extraction from neural networks trained with augmented discretized input

The accuracy of neural networks can be improved when they are trained with discretized continuous attributes as additional inputs. Such input augmentation makes it easier for the network weights to form more accurate decision boundaries when the data samples of different classes in the data set are contained in distinct hyper-rectangular subregions in the original input space. In this paper we present first how a neural network can be trained with augmented discretized inputs. The additional inputs are obtained by simply dividing the original interval of each continuous attribute into subintervals of equal length. Thermometer encoding scheme is used to represent these discretized inputs. The network is then pruned to remove most of the discretized inputs as well as the original continuous attributes as long as the network still achieves a minimum preset accuracy requirement. We then discuss how MofN rules can be extracted from the pruned network by analyzing the activations of the network's hidden units and the weights of the network connections that remain in the pruned network. For data sets that have sample classes defined by relatively complex boundaries, surprisingly simple MofN rules with very good accuracy rates are obtained.