Unsupervised pattern classification by neural networks

A neural network is applied to the unsupervised pattern classification approach. Given a set of data consisting of unlabeled samples from several classes, the task of unsupervised classification is to label every sample in the same class by the same symbol such that the data set is divided into several clusters. We consider the hypothesis that the data set is drawn from a finite mixture of Gaussian distributions. The network architecture is a two-layer feedforward type: the units of the first layer are Gaussians and each correspond to one component of the mixture. The output layer provides the probability density estimation of the mixture. The weighted competitive learning is used to estimate the mean vectors and the non-diagonal covariance matrices of the Gaussian units. The number of Gaussian units in the hidden layer is optimized by informational criteria. Some of the results are reported, and the performance of this approach is evaluated.

[1]  F. R. A. Hopgood,et al.  Machine Intelligence 2 , 1970, The Mathematical Gazette.

[2]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[3]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[4]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[5]  H. Akaike.  A new look at the statistical model identification , 1974 .

[6]  Jack-Gérard Postaire,et al.  Mode boundary detection by relaxation for cluster analysis , 1989, Pattern Recognit..

[7]  Donald Michie,et al.  Machine Intelligence 4 , 1970 .

[8]  Denis Hamad,et al.  Competitive learning neural networks applied to multivariate data set reduction , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[9]  M. P. Windham,et al.  Information-Based Validity Functionals for Mixture Analysis , 1994 .

[10]  J. A. Leonard,et al.  Radial basis function networks for classifying process faults , 1991, IEEE Control Systems.

[11]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[12]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[13]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[14]  Jack-Gérard Postaire,et al.  An Approximate Solution to Normal Mixture Identification with Application to Unsupervised Pattern Classification , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  John E. Moody,et al.  Note on Learning Rate Schedules for Stochastic Optimization , 1990, NIPS.

[16]  Keinosuke Fukunaga,et al.  The Reduced Parzen Classifier , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.