Effects of Sample Size in Classifier

This paper discusses the effect of finite sample size on pa- rameter estimates and their subsequent use in functions. General and parameter-specific expressions for the expected bias and variance of the functions are derived. These expressions are then applied to the Bhattacharyya distance and the analysis of the linear and quadratic classifiers, providing valuable insight into the relationship between the number of features and the number of training samples. Also, because of the functional form of the expressions, we present an empirical ap- proach which will enable asymptotic performance to be accurately es- timated using a very small number of samples. Results are experimen- tally verified using artificial data in controlled cases and using real, high-dimensional data.