A new incremental neural network for simultaneous clustering and classification

Abstract In this paper, an Incremental Neural Network for Classification and Clustering (INNCC) is proposed. The main advantages of this neural network are the linkage between data topology preservation and classes representation by using the cluster posterior probabilities of classes. It is a constructive model without prior conditions such as a suitable number of nodes. A new neuron is inserted when new data are not represented by existing neurons. In training step, both supervised and unsupervised learning are used. The training dataset contains few samples with class labels and several unlabeled ones. The Support Vector Machines (SVM) operates in the training step to assess the INNCC classification result. The proposed approach has been tested on synthetic and real datasets. Obtained results are very promising.

[1]  T. Martínez,et al.  Competitive Hebbian Learning Rule Forms Perfectly Topology Preserving Maps , 1993 .

[2]  Yi Yang,et al.  Discriminative Nonnegative Spectral Clustering with Out-of-Sample Extension , 2013, IEEE Transactions on Knowledge and Data Engineering.

[3]  Feiping Nie,et al.  Nonlinear dimensionality reduction with relative distance comparison , 2009, Neurocomputing.

[4]  Gerald Sommer,et al.  Dynamic Cell Structure Learns Perfectly Topology Preserving Map , 1995, Neural Computation.

[5]  Amel Hebboul,et al.  An incremental parallel neural network for unsupervised classification , 2011, International Workshop on Systems, Signal Processing and their Applications, WOSSPA.

[6]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Shen Furao,et al.  An enhanced self-organizing incremental neural network for online unsupervised learning , 2007, Neural Networks.

[9]  T. Kohonen.  Self-Organized Formation of Correct Feature Maps , 1982 .

[10]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[11]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[12]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[13]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[14]  Nathan Intrator.  Competitive Learning , 2017, Encyclopedia of Machine Learning and Data Mining.

[15]  D. F. Specht,et al.  Probabilistic neural networks for classification, mapping, or associative memory , 1988, IEEE 1988 International Conference on Neural Networks.

[16]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[18]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[19]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[20]  Arindam Banerjee,et al.  Semi-supervised Clustering by Seeding , 2002, ICML.

[21]  Nong Sang,et al.  Using clustering analysis to improve semi-supervised classification , 2013, Neurocomputing.

[22]  Marko Tscherepanow,et al.  A hierarchical ART network for the stable incremental learning of topological structures and associations from noisy data , 2011, Neural Networks.

[23]  Huaguang Zhang,et al.  A fuzzy support vector machine algorithm for classification based on a novel PIM fuzzy clustering method , 2014, Neurocomputing.

[24]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[25]  Ioannis A. Maraziotis,et al.  A semi-supervised fuzzy clustering algorithm applied to gene expression data , 2012, Pattern Recognit..

[26]  Hui Yu,et al.  An Online Incremental Semi-Supervised Learning Method , 2010, J. Adv. Comput. Intell. Intell. Informatics.

[27]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[28]  Aristides Gionis,et al.  Clustering Aggregation , 2005, ICDE.

[29]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[30]  Shiliang Sun,et al.  A review of optimization methodologies in support vector machines , 2011, Neurocomputing.

[31]  A. Ennaji,et al.  An incremental growing neural gas learns topologies , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[32]  Saman K. Halgamuge,et al.  Class structure visualization with semi-supervised growing self-organizing maps , 2008, Neurocomputing.

[33]  Weifu Chen,et al.  Spectral clustering: A semi-supervised approach , 2012, Neurocomputing.

[34]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[35]  Fei Wang,et al.  Robust self-tuning semi-supervised learning , 2007, Neurocomputing.