Noise-enhanced clustering and competitive learning algorithms

Noise can provably speed up convergence in many centroid-based clustering algorithms. This includes the popular k-means clustering algorithm. The clustering noise benefit follows from the general noise benefit for the expectation-maximization algorithm because many clustering algorithms are special cases of the expectation-maximization algorithm. Simulations show that noise also speeds up convergence in stochastic unsupervised competitive learning, supervised competitive learning, and differential competitive learning.

[1]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[2]  Ashok Patel,et al.  2009 Special Issue , 2022 .

[3]  R. Hathaway.  Another interpretation of the EM algorithm for mixture distributions , 1986 .

[4]  Bart Kosko,et al.  Differential competitive learning for centroid estimation and phoneme recognition , 1991, IEEE Trans. Neural Networks.

[5]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[6]  François Chapeau-Blondeau,et al.  Noise-enhanced performance for an optimal Bayesian estimator , 2004, IEEE Transactions on Signal Processing.

[7]  Bart Kosko,et al.  Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence , 1991 .

[8]  B. Kosco.  Differential Hebbian learning , 1987 .

[9]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[10]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[11]  Bart Kosko,et al.  Stochastic competitive learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[12]  Kurt Wiesenfeld,et al.  Controlling Stochastic Resonance , 1999 .

[13]  B. Kosko,et al.  Quantum forbidden-interval theorems for stochastic resonance , 2008, 0801.3141.

[14]  J. Proudfoot,et al.  Noise , 1931, The Indian medical gazette.

[15]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[16]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[17]  Frank Moss,et al.  Noise in human muscle spindles , 1996, Nature.

[18]  G. Parisi,et al.  A Theory of Stochastic Resonance in Climatic Change , 1983 .

[19]  Ashok Patel,et al.  Stochastic Resonance in Continuous and Spiking Neuron Models with Levy Noise , 2008 .

[20]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[21]  B. Kosko,et al.  Adaptive stochastic resonance , 1998, Proc. IEEE.

[22]  Rui Xu,et al.  Clustering Algorithms in Biomedical Research: A Review , 2010, IEEE Reviews in Biomedical Engineering.

[23]  François Chapeau-Blondeau,et al.  Noise-enhanced capacity via stochastic resonance in an asymmetric binary channel , 1997 .

[24]  Greg Hamerly,et al.  Alternatives to the k-means algorithm that find better clusterings , 2002, CIKM '02.

[25]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[26]  I︠a︡. Z. T︠S︡ypkin,et al.  Foundations of the theory of learning systems , 1973 .

[27]  Peter Hänggi,et al.  Stochastic resonance in biology. How noise can enhance detection of weak signals and help improve biological information processing. , 2002, Chemphyschem : a European journal of chemical physics and physical chemistry.

[28]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[29]  Adi R. Bulsara,et al.  Preface , 1993 .

[30]  Bart Kosko,et al.  Virtual Worlds as Fuzzy Cognitive Maps , 1994, Presence: Teleoperators & Virtual Environments.

[31]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[32]  Jong-Sheng Cherng,et al.  A hypergraph based clustering algorithm for spatial data sets , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[33]  Chongwu Zhou,et al.  Noise-Enhanced Detection of Subthreshold Signals With Carbon Nanotubes , 2006, IEEE Transactions on Nanotechnology.

[34]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[35]  Ron Shamir,et al.  A clustering algorithm based on graph connectivity , 2000, Inf. Process. Lett..

[36]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[37]  Gregoire Nicolis,et al.  Stochastic resonance , 2007, Scholarpedia.

[38]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[39]  Sudeshna Sinha,et al.  A noise-assisted reprogrammable nanomechanical logic gate. , 2010, Nano letters.

[40]  V. J. Rayward-Smith,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[41]  Yishay Mansour,et al.  An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clustering , 1997, UAI.

[42]  Osonde Osoba,et al.  Noise benefits in the expectation-maximization algorithm: Nem theorems and models , 2011, The 2011 International Joint Conference on Neural Networks.

[43]  Bart Kosko,et al.  Stochastic resonance in noisy threshold neurons , 2003, Neural Networks.

[44]  Adi R. Bulsara,et al.  Tuning in to Noise , 1996 .

[45]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[46]  J. Javier Brey,et al.  STOCHASTIC RESONANCE IN A ONE-DIMENSIONAL ISING MODEL , 1996 .

[47]  Thomas J. Watson,et al.  An empirical study of the naive Bayes classifier , 2001 .

[48]  David G. Stork,et al.  Pattern Classification , 1973 .

[49]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[50]  Pramod K. Varshney,et al.  Noise Enhanced Nonparametric Detection , 2009, IEEE Transactions on Information Theory.

[51]  Ashok Patel,et al.  Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation , 2010, IEEE Signal Processing Letters.

[52]  B. Kosko.  Differential Hebbian learning , 2008 .

[53]  Osonde A. Osoba,et al.  The Noisy Expectation Maximization Algorithm , 2013 .

[54]  Carson C. Chow,et al.  Stochastic resonance without tuning , 1995, Nature.

[55]  Ashok Patel,et al.  Noise Benefits in Quantizer-Array Correlation Detection and Watermark Decoding , 2011, IEEE Transactions on Signal Processing.

[56]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[57]  C. Pearce,et al.  Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization , 2008 .

[58]  R. D. Boss,et al.  Noise effects in an electronic model of a single neuron , 1989, Biological Cybernetics.

[59]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[60]  Tim Sauer,et al.  Chaotic Stochastic Resonance: Noise-Enhanced Reconstruction of Attractors , 1997 .

[61]  Bulsara,et al.  Threshold detection of wideband signals: A noise-induced maximum in the mutual information. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[62]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[63]  Hänggi,et al.  Nonlinear quantum stochastic resonance. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[64]  Bart Kosko,et al.  Fuzzy prediction and filtering in impulsive noise , 1996, Fuzzy Sets Syst..

[65]  James C. Bezdek,et al.  Clustering with a genetically optimized approach , 1999, IEEE Trans. Evol. Comput..

[66]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .