A novel modular RBF neural network based on a brain-like partition method

In this study, a modular design methodology inherited from cognitive neuroscience and neurophysiology is proposed to develop artificial neural networks, aiming to realize the powerful capability of brain—divide and conquer—when tackling complex problems. First, a density-based brain-like partition method is developed to construct the modular architecture, with a highly connected center in each sub-network as the human brain. The whole task is also divided into different sub-tasks at this stage. Then, a compact radial basis function (RBF) network with fast learning speed and desirable generalization performance is applied as the sub-network to solve the corresponding task. On the one hand, the modular structure helps to improve the ability of neural networks on complex problems by implementing divide and conquer. On the other hand, sub-networks with considerable ability could guarantee the parsimonious and generalization of the entire neural network. Finally, the novel modular RBF (NM-RBF) network is evaluated through multiple benchmark numerical experiments, and results demonstrate that the NM-RBF network is capable of constructing a relative compact architecture during a short learning process with achievable satisfactory generalization performance, showing its effectiveness and outperformance.

[1]  Yi Ding,et al.  A modular neural network architecture with concept , 2014, Neurocomputing.

[2]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[3]  C. Hilgetag,et al.  Hierarchical modular brain connectivity is a stretch for criticality , 2014, Trends in Cognitive Sciences.

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Karl J. Friston,et al.  Structural and Functional Brain Networks: From Connections to Cognition , 2013, Science.

[6]  H. Chris Tseng,et al.  Modular neural networks with applications to pattern profiling problems , 2009, Neurocomputing.

[7]  Bogdan M. Wilamowski,et al.  PolyNet: A Polynomial-Based Learning Machine for Universal Approximation , 2015, IEEE Transactions on Industrial Informatics.

[8]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[9]  Sung-Kwun Oh,et al.  Optimized face recognition algorithm using radial basis function neural networks and its practical applications , 2015, Neural Networks.

[10]  Hao Yu,et al.  Fast and Efficient Second-Order Method for Training Radial Basis Function Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Zhiming Li,et al.  Modeling and Monitoring of Dynamic Processes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[12]  B.M. Wilamowski,et al.  Neural network architectures and learning algorithms , 2009, IEEE Industrial Electronics Magazine.

[13]  Tetsuo Furukawa,et al.  Modular network SOM , 2009, Neural Networks.

[14]  Jean-Charles Créput,et al.  A massively parallel neural network approach to large-scale Euclidean traveling salesman problems , 2017, Neurocomputing.

[15]  Shengli Wu,et al.  Effective Neural Network Ensemble Approach for Improving Generalization Performance , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Alexander V. Goltsev,et al.  Modular neural networks with Hebbian learning rule , 2009, Neurocomputing.

[17]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[18]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[19]  Fevrier Valdez,et al.  Modular Neural Networks architecture optimization with a new nature inspired method using a fuzzy combination of Particle Swarm Optimization and Genetic Algorithms , 2014, Information Sciences.

[20]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Hao Yu,et al.  Selection of Proper Neural Network Sizes and Architectures—A Comparative Study , 2012, IEEE Transactions on Industrial Informatics.

[22]  Hai Zhao,et al.  Parallelized extreme learning machine ensemble based on min-max modular network , 2014, Neurocomputing.

[23]  Junfei Qiao,et al.  An online self-adaptive modular neural network for time-varying systems , 2014, Neurocomputing.

[24]  Yuichi Motai,et al.  Multicolumn RBF Network , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Gang Feng,et al.  A Direct Self-Constructing Neural Controller Design for a Class of Nonlinear Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Wei Lu,et al.  An Adaptive-PSO-Based Self-Organizing RBF Neural Network , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[28]  Nasser L. Azad,et al.  A modular ridge randomized neural network with differential evolutionary distributor applied to the estimation of sea ice thickness , 2016, Soft Computing.

[29]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[30]  Hao Yu,et al.  An Incremental Design of Radial Basis Function Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Patricia Melin,et al.  Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure , 2014, Eng. Appl. Artif. Intell..

[32]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[33]  Witold Pedrycz,et al.  Using multilayer perceptrons as receptive fields in the design of neural networks , 2009, Neurocomputing.