Determining the Optimal Number of Clusters by an Extended RPCL Algorithm

Determining an appropriate number of clusters is a diicult yet important problem. The rival penalized competitive learning (RPCL) algorithm was designed to solve this problem. But its performance is not satisfactory when there are overlapped clusters or in the cases where the input vectors contain dependent components. This paper addresses this problem by incorporating full covariance matrices into the original RPCL algorithm. The resulting algorithm, referred to as the extended RPCL algorithm, progressively eliminates the units whose clusters contain only a small portion of the training data. The algorithm is applied to determine the number of clusters of a Gaussian mixture distribution. It is also applied to optimize the architecture of elliptical basis function networks for speaker veriication and for vowel classiication. It is found that the covariance matrices obtained by the extended RPCL algorithm have a better representation of the clusters than that obtained by the original RPCL algorithm, resulting in a lower veriication error rate in the speaker veriication problem and a higher recognition accuracy in the vowel classiication problem.

[1]  Man-Wai Mak,et al.  Maximum likelihood estimation of elliptical basis function parameters with application to speaker verification , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[2]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[3]  Lei Xu,et al.  Rival penalized competitive learning, finite mixture, and multisets clustering , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[4]  Sung Yang Bang,et al.  An Efficient Method to Construct a Radial Basis Function Neural Network Classifier , 1997, Neural Networks.

[5]  Yiu-ming Cheung,et al.  Application of adaptive RPCL-CLP with trading system to foreign exchange investment , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[6]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[7]  Erkki Oja,et al.  Rival penalized competitive learning for clustering analysis, RBF net, and curve detection , 1993, IEEE Trans. Neural Networks.

[8]  Lai-Wan Chan,et al.  Using rival penalized competitive clustering for feature indexing in Hong Kong's textile and fashion image database , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[9]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[10]  Michael R. Berthold,et al.  Boosting the Performance of RBF Networks with Dynamic Decay Adjustment , 1994, NIPS.

[11]  Nicolaos B. Karayiannis,et al.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques , 1997, IEEE Trans. Neural Networks.

[12]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[13]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[14]  M. F. Tenorio,et al.  A ClusNet architecture for prediction , 1993, IEEE International Conference on Neural Networks.

[15]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[16]  Adam Krzyżak,et al.  Unsupervised and supervised classifications by rival penalized competitive learning , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[17]  Sara H. Basson,et al.  NTIMIT: a phonetically balanced, continuous speech, telephone bandwidth speech database , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[18]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[19]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[20]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[21]  Yiu-ming Cheung,et al.  A RPCL-CLP architecture for financial time series forecasting , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[22]  Y. Chien,et al.  Pattern classification and scene analysis , 1974 .

[23]  Man Wai Mak.  Text-Independent Speaker Verification over a Telephone Network by Radial Basis Function Networks , 1996 .