Adaptive Resonance Theory-based Topological Clustering with a Divisive Hierarchical Structure Capable of Continual Learning

Thanks to an ability for handling the plasticity-stability dilemma, Adaptive Resonance Theory (ART) is considered as an effective approach for realizing continual learning. In general, however, the clustering performance of ART-based algorithms strongly depends on a similarity threshold, i.e., a vigilance parameter, which is data-dependent and specified by hand. This paper proposes an ART-based topological clustering algorithm with a mechanism that automatically estimates a similarity threshold from a distribution of data points. In addition, for the improving information extraction performance, a divisive hierarchical clustering algorithm capable of continual learning is proposed by introducing a hierarchical structure to the proposed algorithm. Simulation experiments show that the proposed algorithm shows the comparative clustering performance compared with recently proposed hierarchical clustering algorithms. INDEX TERMS Adaptive Resonance Theory, Topological Clustering, Hierarchical Clustering, Continual Learning.

[1]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[2]  Hisao Ishibuchi,et al.  Divisive Hierarchical Clustering Based on Adaptive Resonance Theory , 2020, 2020 International Symposium on Community-centric Systems (CcS).

[3]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[4]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[5]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[6]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[7]  P. Deb.  Finite Mixture Models , 2008 .

[8]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[9]  Daniel Boley,et al.  Principal Direction Divisive Partitioning , 1998, Data Mining and Knowledge Discovery.

[10]  Chu Kiong Loo,et al.  Kernel Bayesian ART and ARTMAP , 2018, Neural Networks.

[11]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[12]  Marko Tscherepanow,et al.  TopoART: A Topology Learning Hierarchical ART Network , 2010, ICANN.

[13]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[14]  Christopher F. Parmeter,et al.  Normal reference bandwidths for the general order, multivariate kernel density derivative estimator , 2012 .

[15]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[16]  Donald C. Wunsch,et al.  Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence , 2018, Neural Networks.

[17]  Ajalmar R. da Rocha Neto,et al.  OnARTMAP: A Fuzzy ARTMAP-based Architecture , 2018, Neural Networks.

[18]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[19]  Chin-Teng Lin,et al.  A review of clustering techniques and developments , 2017, Neurocomputing.

[20]  Giansalvo Cirrincione,et al.  The GH-EXIN neural network for hierarchical clustering , 2020, Neural Networks.

[21]  Ajalmar R. da Rocha Neto,et al.  A novel fuzzy ARTMAP with area of influence , 2020, Neurocomputing.

[22]  Marzuki Khalid,et al.  Evolutionary Fuzzy ARTMAP Neural Networks for Classification of Semiconductor Defects , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Pasi Fränti,et al.  K-means properties on six clustering benchmark datasets , 2018, Applied Intelligence.

[24]  Hisao Ishibuchi,et al.  Fast Topological Adaptive Resonance Theory Based on Correntropy Induced Metric , 2019, 2019 IEEE Symposium Series on Computational Intelligence (SSCI).

[25]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[26]  Benjamin Moseley,et al.  An Objective for Hierarchical Clustering in Euclidean Space and Its Connection to Bisecting K-means , 2020, AAAI.

[27]  Boaz Lerner,et al.  The Bayesian ARTMAP , 2007, IEEE Transactions on Neural Networks.

[28]  Ezequiel López-Rubio,et al.  The Growing Hierarchical Neural Gas Self-Organizing Neural Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Hisao Ishibuchi,et al.  Topological Clustering via Adaptive Resonance Theory With Information Theoretic Learning , 2019, IEEE Access.

[30]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[31]  Daniel Berrar,et al.  A self-organizing incremental neural network for continual supervised learning , 2021, Expert Syst. Appl..

[32]  Stefan Wermter,et al.  Lifelong learning of human actions with deep neural network self-organization , 2017, Neural Networks.

[33]  Daniel Berrar,et al.  SOINN+, a Self-Organizing Incremental Neural Network for Unsupervised Learning from Noisy Data Streams , 2020, Expert Syst. Appl..

[34]  M. Cugmas,et al.  On comparing partitions , 2015 .