AutoClustering: A Feed-Forward Neural Network Based Clustering Algorithm

Since a clustering process can be regarded as a map of data to cluster labels, it should be natural to employ a deep learning technique, especially a feed-forward neural network, to realize the clustering method. In this study, we discussed a novel clustering method realized only by a feed-forward neural network. Unlike self-organizing maps and growing neural gas networks, the proposed method is compatible with deep learning neural networks. The proposed method has three parts: a map of records to clusters (encoder), a map of clusters to their exemplars (decoder), and a loss function to measure positional closeness between the records and the exemplars. In order to accelerate clustering performance, we proposed an improved activation function at the encoder, which migrates a soft-max function to a max function continuously. Though most of the clustering methods require the number of clusters in advance, the proposed method naturally provides the number of clusters as the number of unique one-hot vectors obtained as a result. We also discussed the existence of local minima of the loss function and their relationship to clusters.

[1]  Kadim Tasdemir,et al.  Neural network based approximate spectral clustering for remote sensing images , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[2]  R. D'Andrade.  U-statistic hierarchical clustering , 1978 .

[3]  Alexander L. Tatuzov,et al.  Neural network data clustering on the basis of scale invariant entropy , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[4]  R Herwig,et al.  Comparative gene expression profiling by oligonucleotide fingerprinting. , 1998, Nucleic acids research.

[5]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[6]  Brijesh Verma,et al.  A novel soft cluster neural network for the classification of suspicious areas in digital mammograms , 2009, Pattern Recognit..

[7]  Joydeep Ghosh,et al.  Scale-based clustering using the radial basis function network , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  Julia Hirschberg,et al.  V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure , 2007, EMNLP.

[9]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[10]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[11]  Zsolt Kira,et al.  Neural network-based clustering using pairwise constraints , 2015, ArXiv.

[12]  S. Bandyopadhyay,et al.  Nonparametric genetic clustering: comparison of validity indices , 2001, IEEE Trans. Syst. Man Cybern. Syst..

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Javad Behboodian,et al.  On a mixture of normal distributions , 1970 .

[15]  Ling Guan,et al.  Investigations of the self organising tree map , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[16]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[17]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[18]  Teuvo Kohonen,et al.  Essentials of the self-organizing map , 2013, Neural Networks.

[19]  Sameem Abdul Kareem,et al.  An efficient semisupervised feedforward neural network clustering , 2014, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[20]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[22]  Hesham H. Amin,et al.  Clustering of user activities based on adaptive threshold spiking neural networks , 2017, 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN).

[23]  Sameem Abdul Kareem,et al.  A dynamic semisupervised feedforward neural network clustering , 2017, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[24]  LiMin Fu,et al.  A neural network model for real-time adaptive clustering , 1993, IEEE International Conference on Neural Networks.

[25]  Han Xiao,et al.  Max-Entropy Feed-Forward Clustering Neural Network , 2015, ArXiv.

[26]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[27]  C. Müller,et al.  Large-scale clustering of cDNA-fingerprinting data. , 1999, Genome research.

[28]  Sameem Abdul Kareem,et al.  A SINGLE-LAYER SEMI-SUPERVISED FEED FORWARD NEURAL NETWORK CLUSTERING METHOD , 2015 .

[29]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[30]  Peter Willett,et al.  Comparison of Hierarchie Agglomerative Clustering Methods for Document Retrieval , 1989, Comput. J..

[31]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[32]  Vahid Nourani,et al.  IMPLICATION OF SOM-ANN BASED CLUSTERING FOR MULTI-STATION RAINFALL-RUNOFF MODELING , 2015 .

[33]  Sanghamitra Bandyopadhyay,et al.  Classification and learning using genetic algorithms - applications in bioinformatics and web intelligence , 2007, Natural computing series.

[34]  Zhao Kang,et al.  Twin Learning for Similarity and Clustering: A Unified Kernel Approach , 2017, AAAI.

[35]  Ke-Lin Du,et al.  Clustering: A neural network approach , 2010, Neural Networks.

[36]  Ujjwal Maulik,et al.  Genetic clustering for automatic evolution of clusters and application to image classification , 2002, Pattern Recognit..