Clustering using multilayer perceptrons

Abstract In this paper we present a multilayer perceptron-based approach for data clustering. Traditionally, data clustering is performed using either exemplar-based methods that employ some form of similarity or distance measure, discriminatory function-based methods that attempt to identify one or several cluster-dividing hyper-surfaces, point-by-point associative methods that attempt to form groups of points in a pyramidal manner by directly examining the proximity between pairs of points or groups of points, and probabilistic methods which assume that data are sampled from mixture distributions. Commonly, in exemplar-based methods, each cluster is represented by a multi-dimensional centroid. In this paper, we explore the function approximation capabilities of multilayer perceptron neural networks in order to build exemplars which are not simply points but curves or surfaces. The proposed technique aims to group data points into arbitrary-shaped point clouds. The proposed approach may exhibit problems similar to other traditional exemplar-based clustering techniques such as k -means, including convergence to local minimum solutions with respect to the cost function. However, it is illustrated in this work that approaches such as split-and-merge can be appropriately adjusted and employed in the proposed technique, in order to alleviate the problem of reaching poor local minimum solutions.

[1]  Kenneth Levenberg A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[2]  Enrique H. Ruspini,et al.  A New Approach to Clustering , 1969, Inf. Control..

[3]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[4]  Keinosuke Fukunaga,et al.  A Graph-Theoretic Approach to Nonparametric Cluster Analysis , 1976, IEEE Transactions on Computers.

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  Mahdieh Soleymani Baghshah,et al.  A fuzzy clustering algorithm for finding arbitrary shaped clusters , 2008, 2008 IEEE/ACS International Conference on Computer Systems and Applications.

[7]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[8]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[9]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[10]  Dimitrios Charalampidis,et al.  A modified k-means algorithm for circular invariant clustering , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[13]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[14]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[15]  Chien-Hsing Chou,et al.  Short Papers , 2001 .

[16]  Bidyut Baran Chaudhuri,et al.  A novel multiseed nonhierarchical data clustering technique , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[17]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[18]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[19]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[20]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .