Accuracy on a Hold-out Set: The Red Herring of Data Mining

Abstract: When performing predictive modeling, the overall goal is to generate models likely to have high accuracy when applied to novel data. A technique commonly used to maximize generalization a ...