Competitive Repetition Suppression (CoRe) Clustering: A Biologically Inspired Learning Model With Application to Robust Clustering

Determining a compact neural coding for a set of input stimuli is an issue that encompasses several biological memory mechanisms as well as various artificial neural network models. In particular, establishing the optimal network structure is still an open problem when dealing with unsupervised learning models. In this paper, we introduce a novel learning algorithm, named competitive repetition-suppression (CoRe) learning, inspired by a cortical memory mechanism called repetition suppression (RS). We show how such a mechanism is used, at various levels of the cerebral cortex, to generate compact neural representations of the visual stimuli. From the general CoRe learning model, we derive a clustering algorithm, named CoRe clustering, that can automatically estimate the unknown cluster number from the data without using a priori information concerning the input distribution. We illustrate how CoRe clustering, besides its biological plausibility, posses strong theoretical properties in terms of robustness to noise and outliers, and we provide an error function describing CoRe learning dynamics. Such a description is used to analyze CoRe relationships with the state-of-the art clustering models and to highlight CoRe similitude with rival penalized competitive learning (RPCL), showing how CoRe extends such a model by strengthening the rival penalization estimation by means of loss functions from robust statistics.

[1]  James C. Bezdek,et al.  Fuzzy c-means clustering of incomplete data , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Hichem Frigui,et al.  Unsupervised learning of prototypes and attribute weights , 2004, Pattern Recognit..

[3]  R. Desimone,et al.  Neural mechanisms for visual memory and their role in attention. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Erkki Oja,et al.  Rival penalized competitive learning for clustering analysis, RBF net, and curve detection , 1993, IEEE Trans. Neural Networks.

[5]  Tommy W. S. Chow,et al.  Self-Organizing-Map Based Clustering Using a Local Clustering Validity Index , 2004, Neural Processing Letters.

[6]  Jinwen Ma,et al.  A cost-function approach to rival penalized competitive learning (RPCL) , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  R. O’Reilly,et al.  Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. , 2003, Psychological review.

[8]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[9]  Alex Martin,et al.  Properties and mechanisms of perceptual priming , 1998, Current Opinion in Neurobiology.

[10]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[11]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[12]  Lei Xu,et al.  Data smoothing regularization, multi-sets-learning, and problem solving strategies , 2003, Neural Networks.

[13]  Davide Bacciu,et al.  Competitive Repetition-suppression (CoRe) Learning , 2006, ICANN.

[14]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[15]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[16]  Yiu-ming Cheung,et al.  Rival-Model Penalized Self-Organizing Map , 2007, IEEE Transactions on Neural Networks.

[17]  Michael Gribskov,et al.  Rival penalized competitive learning (RPCL): a topology-determining algorithm for analyzing gene expression data , 2003, Comput. Biol. Chem..

[18]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[19]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[20]  Davide Bacciu,et al.  Convergence Behavior of Competitive Repetition-Suppression Clustering , 2007, ICONIP.

[21]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[22]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[23]  Robert M. French,et al.  Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks , 1992 .

[24]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[25]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[26]  R. Desimone,et al.  Parallel neuronal mechanisms for short-term memory. , 1994, Science.

[27]  Dao-Qiang Zhang,et al.  A novel kernelized fuzzy C-means algorithm with application in medical image segmentation , 2004, Artif. Intell. Medicine.

[28]  Peraphon Sophatsathit,et al.  An Rpcl-Based Indexing Approach For Software Component Classification , 2004, Int. J. Softw. Eng. Knowl. Eng..

[29]  Francesco Camastra,et al.  A novel kernel method for clustering , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[31]  Mohamed A. Ismail,et al.  On-line hierarchical clustering , 1998, Pattern Recognit. Lett..

[32]  Rui J. P. de Figueiredo,et al.  A new neural network for cluster-detection-and-labeling , 1998, IEEE Trans. Neural Networks.

[33]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[34]  Davide Bacciu,et al.  A Robust Bio-Inspired Clustering Algorithm for the Automatic Determination of Unknown Cluster Number , 2007, 2007 International Joint Conference on Neural Networks.

[35]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[36]  Allen Gersho,et al.  Competitive learning and soft competition for vector quantizer design , 1992, IEEE Trans. Signal Process..

[37]  Lei Xu,et al.  A unified perspective and new results on RHT computing, mixture based learning, and multi-learner based problem solving , 2007, Pattern Recognit..

[38]  Zhi-Qiang Liu,et al.  Self-splitting competitive learning: a new on-line clustering paradigm , 2002, IEEE Trans. Neural Networks.

[39]  Ethem Alpaydin,et al.  Constructive feedforward ART clustering networks. I , 2002, IEEE Trans. Neural Networks.

[40]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[41]  Riccardo Rizzo,et al.  A comparison between habituation and conscience mechanism in self-organizing maps , 2006, IEEE Transactions on Neural Networks.

[42]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[43]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[44]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[45]  Andrzej Bargiela,et al.  General fuzzy min-max neural network for clustering and classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[46]  Isabel Gauthier,et al.  Repetition‐induced changes in BOLD response reflect accumulation of neural activity , 2006, Human brain mapping.

[47]  Enrique Vidal,et al.  A R ELATIVE APPROACH TO HIERARCHICAL CLUSTERING , 2000 .

[48]  Giuseppe Acciani,et al.  A feature extraction unsupervised neural network for an environmental data set , 2003, Neural Networks.

[49]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[50]  Miin-Shen Yang,et al.  A similarity-based robust clustering method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  J. Ringo,et al.  Mnemonic Responses of Single Units Recorded from Monkey Inferotemporal Cortex, Accessed via Transcommissural Versus Direct Pathways: A Dissociation between Unit Activity and Behavior , 1996, The Journal of Neuroscience.

[52]  Hichem Frigui,et al.  A Robust Competitive Clustering Algorithm With Applications in Computer Vision , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Aristidis Likas,et al.  Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting , 2007, IEEE Transactions on Neural Networks.