A Kernel Bayesian Adaptive Resonance Theory with A Topological Structure

This paper attempts to solve the typical problems of self-organizing growing network models, i.e. (a) an influence of the order of input data on the self-organizing ability, (b) an instability to high-dimensional data and an excessive sensitivity to noise, and (c) an expensive computational cost by integrating Kernel Bayes Rule (KBR) and Correntropy-Induced Metric (CIM) into Adaptive Resonance Theory (ART) framework. KBR performs a covariance-free Bayesian computation which is able to maintain a fast and stable computation. CIM is a generalized similarity measurement which can maintain a high-noise reduction ability even in a high-dimensional space. In addition, a Growing Neural Gas (GNG)-based topology construction process is integrated into the ART framework to enhance its self-organizing ability. The simulation experiments with synthetic and real-world datasets show that the proposed model has an outstanding stable self-organizing ability for various test environments.

[1]  Chu Kiong Loo,et al.  Growing Neural Gas with Correntropy Induced Metric , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[2]  Amaury Lendasse,et al.  High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications , 2015, IEEE Access.

[3]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[4]  Gail A. Carpenter,et al.  ART-EMAP: A neural network architecture for object recognition by evidence accumulation , 1995, IEEE Trans. Neural Networks.

[5]  Hojjat Adeli,et al.  A New Neural Dynamic Classification Algorithm , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Linqiang Pan,et al.  Spiking Neural P Systems With Communication on Request and Mute Rules , 2017, IEEE Transactions on Parallel and Distributed Systems.

[7]  Fei Xu,et al.  Spiking Neural P Systems with Communication on Request and Polarizations , 2020, Int. J. Unconv. Comput..

[8]  M. Cugmas,et al.  On comparing partitions , 2015 .

[9]  Philipp Cimiano,et al.  Online Semi-Supervised Growing Neural Gas , 2012, Int. J. Neural Syst..

[10]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[11]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[12]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[13]  Yasha Zeinali,et al.  Competitive probabilistic neural network , 2017, Integr. Comput. Aided Eng..

[14]  Sameem Abdul Kareem,et al.  Review of current Online Dynamic Unsupervised Feed Forward Neural Network classification , 2014 .

[15]  Shen Furao,et al.  An enhanced self-organizing incremental neural network for online unsupervised learning , 2007, Neural Networks.

[16]  Sergei Vassilvitskii,et al.  How slow is the k-means method? , 2006, SCG '06.

[17]  Chu Kiong Loo,et al.  Kernel Bayesian ART and ARTMAP , 2018, Neural Networks.

[18]  Andrew J. Hanson,et al.  Geometry for N-Dimensional Graphics , 1994, Graphics Gems.

[19]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[20]  Le Song,et al.  Kernel Bayes' rule: Bayesian inference with positive definite kernels , 2013, J. Mach. Learn. Res..

[21]  Yaxin Peng,et al.  Nonlinear Semi-Supervised Metric Learning Via Multiple Kernels and Local Topology , 2017, Int. J. Neural Syst..

[22]  Babak Mokhtarani,et al.  Prediction of shockwave location in supersonic nozzle separation using self-organizing map classification and artificial neural network modeling , 2016 .

[23]  C. L. Philip Chen,et al.  Data-intensive applications, challenges, techniques and technologies: A survey on Big Data , 2014, Inf. Sci..

[24]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[25]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[26]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[27]  Retantyo Wardoyo,et al.  Time Complexity Analysis of Support Vector Machines (SVM) in LibSVM , 2015 .

[28]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[29]  Robert F. Harrison,et al.  A modified fuzzy ARTMAP architecture for the approximation of noisy mappings , 1995, Neural Networks.

[30]  Zahir Tari,et al.  A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis , 2014, IEEE Transactions on Emerging Topics in Computing.

[31]  José Carlos Príncipe,et al.  Self-organizing maps with information theoretic learning , 2015, Neurocomputing.

[32]  Ferrante Neri,et al.  An Optimization Spiking Neural P System for Approximately Solving Combinatorial Optimization Problems , 2014, Int. J. Neural Syst..

[33]  Ezequiel López-Rubio,et al.  Bregman Divergences for Growing Hierarchical Self-Organizing Networks , 2014, Int. J. Neural Syst..

[34]  Hadi Meidani,et al.  Deep Learning for Accelerated Seismic Reliability Analysis of Transportation Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[35]  Hojjat Adeli,et al.  Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete , 2017 .

[36]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[37]  Boaz Lerner,et al.  The Bayesian ARTMAP , 2007, IEEE Transactions on Neural Networks.

[38]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[39]  Miguel A. Molina-Cabello,et al.  Foreground Detection by Competitive Learning for Varying Input Distributions , 2017, Int. J. Neural Syst..

[40]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[41]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[42]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[43]  Hossein Hashemi,et al.  End‐to‐End Deep Learning Methodology for Real‐Time Traffic Network Management , 2018, Comput. Aided Civ. Infrastructure Eng..

[44]  Mo Hai.  大数据聚类算法综述 (Survey of Clustering Algorithms for Big Data) , 2016, 计算机科学.

[45]  Shen Furao,et al.  A fast nearest neighbor classifier based on self-organizing incremental neural network , 2008, Neural Networks.

[46]  Osamu Hasegawa,et al.  Nonparametric Density Estimation Based on Self-Organizing Incremental Neural Network for Large Noisy Data , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Marko Tscherepanow,et al.  TopoART: A Topology Learning Hierarchical ART Network , 2010, ICANN.

[48]  Yi-Zhou Lin,et al.  Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning , 2017, Comput. Aided Civ. Infrastructure Eng..

[49]  R. Khabibullin,et al.  An algorithm for constructing high dimensional distributions from distributions of lower dimension , 2014 .

[50]  Thomas Burwick,et al.  Optimal Algorithmic Complexity of Fuzzy ART , 1998, Neural Processing Letters.

[51]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[52]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[53]  Ezequiel López-Rubio,et al.  Improving the Quality of Self-Organizing Maps by Self-Intersection Avoidance , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[55]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[56]  Boguslaw Cyganek,et al.  Image recognition with deep neural networks in presence of noise - Dealing with and taking advantage of distortions , 2017, Integr. Comput. Aided Eng..

[57]  Andrew G. Lamperski,et al.  Seizure Control in a Computational Model Using a Reinforcement Learning Stimulation Paradigm , 2017, Int. J. Neural Syst..

[58]  Stephen Grossberg,et al.  From Interactive Activation to Adaptive Resonance , 1987 .

[59]  Katya Scheinberg,et al.  Efficient SVM Training Using Low-Rank Kernel Representations , 2002, J. Mach. Learn. Res..

[60]  Hojjat Adeli,et al.  Enhanced probabilistic neural network with local decision circles: A robust classifier , 2010, Integr. Comput. Aided Eng..

[61]  M. Wand.  Fast Computation of Multivariate Kernel Estimators , 1994 .

[62]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[63]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.