A survey on hyper basis function neural networks

ABSTRACT Hyper basis function neural networks (HBFNNs) have gained considerable attention in recent years, which have shown good performance in a variety of application domains. In this paper, we first briefly introduce the development of neural networks. Then the structure of HBFNNs is presented in detail. HBFNNs are an extension of radial basis function neural networks (RBFNNs), which use the weighted norm instead of the Euclidean norm to represent the distance from input data to hidden layer neuron centres. With this change, the generalization ability of neural networks becomes stronger than that of RBFNNs. Subsequently, we summarize several commonly used training methods for HBFNNs, including static training methods and dynamic training methods. Finally, we give several typical application fields of HBFNNs.

[1]  Yu Guo,et al.  Adaptive neural network predictive control for nonlinear pure feedback systems with input delay , 2012 .

[2]  Paul S. Bradley,et al.  Refining Initial Points for K-Means Clustering , 1998, ICML.

[3]  Anil K. Jain,et al.  Comments on "A self-organizing network for hyperellipsoidal clustering (HEC)" [and reply] , 1997, IEEE Trans. Neural Networks.

[4]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[5]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[6]  Nicolaos B. Karayiannis,et al.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques , 1997, IEEE Trans. Neural Networks.

[7]  A. Hossein Sahoolizadeh,et al.  A New Face Recognition Method using PCA, LDA and Neural Network , 2008 .

[8]  V. Vasu,et al.  Navigation of non-holonomic mobile robot using neuro-fuzzy logic with integrated safe boundary algorithm , 2017, Int. J. Autom. Comput..

[9]  Anil K. Jain,et al.  A self-organizing network for hyperellipsoidal clustering (HEC) , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  Luc De Raedt,et al.  Proceedings of the 22nd international conference on Machine learning , 2005 .

[11]  Tai C Yang,et al.  Networked control system: a brief survey , 2006 .

[12]  António Abelha,et al.  Prediction of the quality of public water supply using artificial neural networks , 2012 .

[13]  Paramasivan Saratchandran,et al.  Improved GAP-RBF network for classification problems , 2007, Neurocomputing.

[14]  Eric C. Rouchka,et al.  Feature Selection in Cancer Classification from mRNA Data Based on Localized Dimension Reduction , 2009, 2009 International Conference on Machine Learning and Applications.

[15]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[17]  Dipti Srinivasan,et al.  Neural Networks for Real-Time Traffic Signal Control , 2006, IEEE Transactions on Intelligent Transportation Systems.

[18]  Paramasivan Saratchandran,et al.  Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm , 1998, IEEE Trans. Neural Networks.

[19]  Guohui Tian,et al.  A Selective Attention Guided Initiative Semantic Cognition Algorithm for Service Robot , 2018, Int. J. Autom. Comput..

[20]  Eric C. Rouchka,et al.  Reduced HyperBF Networks: Regularization by Explicit Complexity Reduction and Scaled Rprop-Based Training , 2011, IEEE Transactions on Neural Networks.

[21]  Xin-bo Zhang,et al.  A New Validity Index of Fuzzy c-Means Clustering , 2009, 2009 International Conference on Intelligent Human-Machine Systems and Cybernetics.

[22]  Andrzej Cichocki,et al.  Hyper Radial Basis Function Neural Networks for Interference Cancellation with Nonlinear Processing of Reference Signal , 2001, Digit. Signal Process..

[23]  Yaping Zhu,et al.  Design of fault detection observer based on Hyper Basis Function , 2015 .

[24]  Yong-Zhong Lu,et al.  A novel face recognition algorithm for distinguishing faces with various angles , 2008, Int. J. Autom. Comput..

[25]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .

[26]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[27]  Kin Keung Lai,et al.  Credit risk assessment with a multistage neural network ensemble learning approach , 2008, Expert Syst. Appl..

[28]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[29]  Rabia Jafri,et al.  A Survey of Face Recognition Techniques , 2009, J. Inf. Process. Syst..

[30]  Narasimhan Sundararajan,et al.  Real-time performance evaluation of the minimal radial basis function network for identification of time varying nonlinear systems , 2002, Comput. Electr. Eng..

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Xuli Han,et al.  Constructive Approximation to Multivariate Function by Decay RBF Neural Network , 2010, IEEE Transactions on Neural Networks.

[33]  Najdan Vukovic,et al.  A growing and pruning sequential learning algorithm of hyper basis function neural network for function approximation , 2013, Neural Networks.

[34]  Chen Wu,et al.  An Online Self-Adaption Learning Algorithm for Hyper Basis Function Neural Network , 2018, 2018 2nd IEEE Advanced Information Management,Communicates,Electronic and Automation Control Conference (IMCEC).

[35]  Banupriya,et al.  SURVEY ON FACE RECOGNITION , 2014 .

[36]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[37]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[39]  Adnan Acan,et al.  FACE RECOGNITION USING PCA, LDA AND ICA APPROACHES ON COLORED IMAGES , 2003 .

[40]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[41]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[42]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[43]  Xinying Xu,et al.  Deep Learning Based Single Image Super-resolution: A Survey , 2018, International Journal of Automation and Computing.

[44]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[45]  U. Ruckert,et al.  Digital hardware realization of a hyper basis function network for on-line learning , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[46]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[47]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[48]  Carla E. Brodley,et al.  Proceedings of the twenty-first international conference on Machine learning , 2004, International Conference on Machine Learning.

[49]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[50]  Mo-Yuen Chow,et al.  Networked Control System: Overview and Research Trends , 2010, IEEE Transactions on Industrial Electronics.

[51]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Pekka Nieminen,et al.  Prospective and randomised public‐health trial on neural network‐assisted screening for cervical cancer in Finland: Results of the first year , 2003, International journal of cancer.

[53]  Min Tan,et al.  Sliding Mode Control for Flexible-link Manipulators Based on Adaptive Neural Networks , 2018, Int. J. Autom. Comput..

[54]  C. L. Philip Chen,et al.  Broad learning system: A new learning paradigm and system without going deep , 2017, 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[55]  Anil K. Jain,et al.  A self-organizing network for hyperellipsoidal clustering (HEC) , 1996, IEEE Trans. Neural Networks.

[56]  Nikos A. Vlassis,et al.  The global k-means clustering algorithm , 2003, Pattern Recognit..

[57]  Léon Bottou,et al.  Stochastic Gradient Descent Tricks , 2012, Neural Networks: Tricks of the Trade.

[58]  Xin-Ping Guan,et al.  Neural network observer-based networked control for a class of nonlinear systems , 2014, Neurocomputing.

[59]  Bin Jiang,et al.  Fault detection and estimation for a class of nonlinear systems based on neural network observer , 2016, 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC).

[60]  Lianwen Jin,et al.  A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA plus LDA , 2006 .

[61]  Chris H. Q. Ding,et al.  K-means clustering via principal component analysis , 2004, ICML.

[62]  Jingfei Jiang,et al.  An FPGA-based accelerator implementation for deep convolutional neural networks , 2015, 2015 4th International Conference on Computer Science and Network Technology (ICCSNT).

[63]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[64]  Mu-Chun Su,et al.  Application of neural networks using quadratic junctions in cluster analysis , 2001, Neurocomputing.

[65]  Shuicheng Yan,et al.  A survey on deep learning-based fine-grained object classification and semantic segmentation , 2017, International Journal of Automation and Computing.

[66]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[67]  Kazuo Asakawa,et al.  Stock market prediction system with modular neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[68]  Sung-Bae Cho,et al.  Radial basis function neural networks: a topical state-of-the-art survey , 2016, Open Comput. Sci..

[69]  Zohir Dibi,et al.  Modeling of a Smart Nano Force Sensor Using Finite Elements and Neural Networks , 2020, Int. J. Autom. Comput..

[70]  M. Bertero.  Regularization methods for linear inverse problems , 1986 .

[71]  Narasimhan Sundararajan,et al.  Adaptive nonlinear system identification using minimal radial basis function neural networks , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.