About Learning Space and Non-Monotonicity in Assessment of Classification Risk

The characteristic of modern pattern classification methods is to admit empirical risk non-zero, whereas inseparable feature set never provides a chance for learning algorithms to make a classifier with zero of empirical risk. In order to investigate the potential connection between inseparable feature set, which is usually thought as trustless on intuition, and the modern idea on learning problem, this paper argues the necessary condition of the availability of inseparable feature set, by which elaborates an opportunistic learning method to validate the sufficient condition experimentally. Experimental evidences show that inseparable feature subset can make important contributions for improving the performance of pattern classifier. Further more, the relation between the assessment of classification and the predictive performance on test set is proved to be non-monotone in experiments. Both the analytical results and experimental studies reflect that this conclusion may be a challenge to pattern classification and its applications in the future.