Review on Methods to Fix Number of Hidden Neurons in Neural Networks

This paper reviews methods to fix a number of hidden neurons in neural networks for the past 20 years. And it also proposes a new method to fix the hidden neurons in Elman networks for wind speed prediction in renewable energy systems. The random selection of a number of hidden neurons might cause either overfitting or underfitting problems. This paper proposes the solution of these problems. To fix hidden neurons, 101 various criteria are tested based on the statistical errors. The results show that proposed model improves the accuracy and minimal error. The perfect design of the neural network based on the selection criteria is substantiated using convergence theorem. To verify the effectiveness of the model, simulations were conducted on real-time wind data. The experimental results show that with minimum errors the proposed approach can be used for wind speed prediction. The survey has been made for the fixation of hidden neurons in neural networks. The proposed model is simple, with minimal error, and efficient for fixation of hidden neurons in Elman networks.

[1]  Vladik Kreinovich,et al.  Estimates of the Number of Hidden Units and Variation with Respect to Half-Spaces , 1997, Neural Networks.

[2]  Min Han,et al.  The hidden neurons selection of the wavelet networks using support vector machines and ridge regression , 2008, Neurocomputing.

[3]  Panos J. Antsaklis,et al.  A simple method to derive bounds on the size and to train multilayer neural networks , 1991, IEEE Trans. Neural Networks.

[4]  Shin'ichi Tamura,et al.  Capabilities of a four-layered feedforward neural network: four layers versus three , 1997, IEEE Trans. Neural Networks.

[5]  Stephan Trenn,et al.  Multilayer Perceptrons: Approximation Order and Necessary Number of Hidden Units , 2008, IEEE Transactions on Neural Networks.

[6]  Yong Huang,et al.  Convergence Study in Extended Kalman Filter-Based Training of Recurrent Neural Networks , 2011, IEEE Transactions on Neural Networks.

[7]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[8]  Masafumi Hagiwara,et al.  A simple and effective method for removal of hidden units and weights , 1994, Neurocomputing.

[9]  Daniel S. Yeung,et al.  Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure , 2006, Neurocomputing.

[10]  Yuan-Kang Wu,et al.  A literature review of wind forecasting technology in the world , 2007, 2007 IEEE Lausanne Power Tech.

[11]  F. L. Xiong,et al.  A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy , 2003 .

[12]  Yue Liu,et al.  Optimizing number of hidden neurons in neural networks , 2007, Artificial Intelligence and Applications.

[13]  Qing Cao,et al.  Forecasting wind speed with recurrent neural networks , 2012, Eur. J. Oper. Res..

[14]  Takashi Onoda,et al.  Neural network information criterion for the optimal number of hidden units , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  Yixian Yang,et al.  Bounds on the number of hidden neurons in three-layer binary neural networks , 2003, Neural Networks.

[16]  Guang-Bin Huang,et al.  Neuron selection for RBF neural network classifier based on data structure preserving criterion , 2005, IEEE Transactions on Neural Networks.

[17]  Osamu Fujita,et al.  Statistical estimation of the number of hidden units for feedforward neural networks , 1998, Neural Networks.

[18]  Jamal Ahmad Dargham,et al.  Finding the number of hidden neurons for an MLP neural network using coarse to fine search technique , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[19]  Sheela Rani.B.  Role of Hidden Neurons in an Elman Recurrent Neural Network in Classification of Cavitation Signals , 2012 .

[20]  Masahiko Arai,et al.  Bounds on the number of hidden units in binary-valued three-layer neural networks , 1993, Neural Networks.

[21]  Bruce Curry,et al.  Model selection in Neural Networks: Some difficulties , 2006, Eur. J. Oper. Res..

[22]  Deok-Hwan Kim,et al.  Solving local minima problem with large number of hidden nodes on two-layered feed-forward artificial neural networks , 2008, Neurocomputing.

[23]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[24]  Gaurang Panchal,et al.  Behaviour Analysis of Multilayer Perceptrons with Multiple Hidden Neurons and Hidden Layers , 2011 .

[25]  Shuxiang Xu,et al.  A novel approach for determining the optimal number of hidden layer neurons for FNN’s and its application in data mining , 2008 .

[26]  John Bird,et al.  Higher engineering mathematics , 1993 .

[27]  Hao Yu,et al.  Selection of Proper Neural Network Sizes and Architectures—A Comparative Study , 2012, IEEE Transactions on Industrial Informatics.

[28]  Jie Zhang,et al.  A Sequential Learning Approach for Single Hidden Layer Neural Networks , 1998, Neural Networks.

[29]  Mohammad Reza Meybodi,et al.  Backpropagation Algorithm Adaptation Parameters Using Learning Automata , 2001, Int. J. Neural Syst..

[30]  Katsunari Shibata,et al.  Effect of number of hidden neurons on learning in large-scale layered neural networks , 2009, 2009 ICCAS-SICE.

[31]  Yuan Lan,et al.  Constructive hidden nodes selection of extreme learning machine for regression , 2010, Neurocomputing.

[32]  T.W.S. Chow,et al.  The estimation theory and optimization algorithm for the number of hidden units in the higher-order feedforward neural network , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[33]  Jian Wang,et al.  The Upper Bound on the Number of Hidden Neurons in Multi-Valued Multi-Threshold Neural Networks , 2008, 2009 International Workshop on Intelligent Systems and Applications.

[34]  Kenji Nakayama,et al.  Estimation of initial weights and hidden units for fast learning of multilayer neural networks for pattern classification , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[35]  S. Sumathi,et al.  Introduction to neural networks using MATLAB 6.0 , 2006 .

[36]  Jinchuan Ke,et al.  Empirical Analysis of Optimal Hidden Neurons in Neural Network Modeling for Stock Prediction , 2008, 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application.

[37]  Chengxiong Mao,et al.  Wind speed prediction based on the Elman recursion neural networks , 2010, Proceedings of the 2010 International Conference on Modelling, Identification and Control.

[38]  Guang-Bin Huang,et al.  Learning capability and storage capacity of two-hidden-layer feedforward networks , 2003, IEEE Trans. Neural Networks.

[39]  Jeff Heaton,et al.  Introduction to neural networks for C , 2008 .

[40]  Jianye Sun.  Learning algorithm and hidden node selection scheme for local coupled feedforward neural network classifier , 2012, Neurocomputing.

[41]  Whei-Min Lin,et al.  A New Elman Neural Network-Based Control Algorithm for Adjustable-Pitch Variable-Speed Wind-Energy Conversion Systems , 2011, IEEE Transactions on Power Electronics.

[42]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.