A statistical neural network for high-dimensional vector classification

The minimum number of misclassifications in a multi-class classifier is reached when the borders between classes are set according to the Bayes criterion. Unfortunately, this criterion necessitates the knowledge of the probability density function of each class of data, which is unknown in practical problems. The theory of kernel estimators (Parzen windows) provides a way to estimate these probability densities, given a set of data in each class. The computational complexity of these estimators is however much too large in most practical applications; the authors propose here a neural network aimed to estimate the probability density function underlying a set of data, in a sub-optimal way (while performances are quite similar to those in the optimal case), but with a strongly reduced complexity which makes the method useful in practical situations. The algorithm is based on a "competitive learning" vector quantization of the data, and on the choice of optimal widths for the kernels. the authors study the influence of this factor on the classification error rate, and provide examples of the use of the algorithm on real-world data.