Test Sample Accuracy Scales with Training Sample Density in Neural Networks

Intuitively, one would expect the accuracy of a trained neural network’s prediction on a test sample to correlate with how densely that sample is surrounded by seen training samples in representation space. In this work we provide theory and experiments that support this hypothesis. We propose an error function for piecewise linear neural networks that takes a local region in the network’s input space and outputs smooth empirical training error, which is an average of empirical training errors from other regions weighted by network representation distance. A bound on the expected smooth error for each region scales inversely with training sample density in representation space. Empirically, we verify this bound is a strong predictor of the inaccuracy of the network’s prediction on test samples. For unseen test sets, including those with out-of-distribution samples, ranking test samples by their local region’s error bound and discarding samples with the highest bounds raises prediction accuracy by up to 20% in absolute terms, on image classification datasets.