How to make a neural network say "Don't know"

Abstract Despite various advantages, due to improper training, sometimes a Multi-Layer Perceptron (MLP) classifies a test data point far from the training data into a completely irrelevant class. On the other hand, for example, when we train an MLP to distinguish between four types of childhood cancers (neuroblastoma, rhabdomyosarcoma, non-Hodgkin lymphomas, and Ewing sarcoma) using gene expression profiles and test it on some other kind of cancer (or normal patient) data, the test data will be misclassified to one of the four classes. These unexpected situations arise due to the “open world” nature of the problem. Such problems exist with many other learning systems. We want to address these problems by equipping the network with the ability to not make any judgment when it should not. We have developed an algorithm to provide a practical solution to this problem. We first estimate the domain of the training data (sampling window). We show consistency of our estimate along with some other interesting results. An MLP should say “Don’t know” if a test point is outside the sampling window. To realize this, we generate observations from the complement region and label them to a new class “Don’t know”. We train a network with training data along with the generated data to get a classifier that supports the extra class. The problem of generating a large number of points from the complement region is dealt with a novel scheme that exploits the input sensitivity of the system on the complement region via a regularization. We study the effectiveness of our proposed method with several data sets.