Feature Extraction for One-Class Classification

Feature reduction is often an essential part of solving a classification task. One common approach for doing this, is Principal Component Analysis. There the low variance directions in the data are removed and the high variance directions are retained. It is hoped that these high variance directions contain information about the class differences. For one-class classification or novelty detection, the classification task contains one ill-determined class, for which (almost) no information is available. In this paper we show that for one-class classification, the low-variance directions are most informative, and that in the feature reduction a bias-variance trade-off has to be considered which causes that retaining the high variance directions is often not optimal.