Approximation-based Estimation of Learning Rate for Error Back Propagation Algorithm

The paper presents a new method for improvement of the Error Back Propagation, one of the most popular algorithms for training artificial neural networks, that is based on the estimation of the learning rate by the approximation of the error of the output error. Experimental studies confirming the effectiveness of the applied method of improving the network learning effectiveness have been presented

[1]  P. M. Menghal,et al.  Real time control of induction motor using neural network , 2018, 2018 International Conference on Communication information and Computing Technology (ICCICT).

[2]  Ryan P. Adams,et al.  Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks , 2015, ICML.

[3]  Hao Yu,et al.  Selection of Proper Neural Network Sizes and Architectures—A Comparative Study , 2012, IEEE Transactions on Industrial Informatics.

[4]  D. Bui,et al.  A comparative assessment of support vector regression, artificial neural networks, and random forests for predicting and mapping soil organic carbon stocks across an Afromontane landscape. , 2015 .

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  P. Rozycki,et al.  Estimation of deep neural networks capabilities based on a trigonometric approach , 2016, 2016 IEEE 20th Jubilee International Conference on Intelligent Engineering Systems (INES).

[7]  Yongdong Zhang,et al.  Supervised Hash Coding With Deep Neural Network for Environment Perception of Intelligent Vehicles , 2018, IEEE Transactions on Intelligent Transportation Systems.

[8]  Shuhui Li,et al.  Training Recurrent Neural Networks With the Levenberg–Marquardt Algorithm for Optimal Control of a Grid-Connected Converter , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[10]  Taher Niknam,et al.  Probabilistic Load Forecasting Using an Improved Wavelet Neural Network Trained by Generalized Extreme Learning Machine , 2018, IEEE Transactions on Smart Grid.

[11]  Alireza Bahadori,et al.  Determination of oil well production performance using artificial neural network (ANN) linked to the particle swarm optimization (PSO) tool , 2015 .

[12]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[13]  Michael J. Watts,et al.  IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS Publication Information , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Pierluigi Siano,et al.  A Novel RBF Training Algorithm for Short-Term Electric Load Forecasting and Comparative Studies , 2015, IEEE Transactions on Industrial Electronics.

[15]  Min Gan,et al.  A Variable Projection Approach for Efficient Estimation of RBF-ARX Model , 2015, IEEE Transactions on Cybernetics.

[16]  Min Gan,et al.  A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling , 2010, Inf. Sci..

[17]  Prabhat,et al.  Artificial Neural Network , 2018, Encyclopedia of GIS.

[18]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[19]  Long Zhang,et al.  A New Extension of Newton Algorithm for Nonlinear System Modelling Using RBF Neural Networks , 2013, IEEE Transactions on Automatic Control.

[20]  Wei-Chang Yeh,et al.  New Parameter-Free Simplified Swarm Optimization for Artificial Neural Network Training and its Application in the Prediction of Time Series , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Hao Yu,et al.  Parity- N Problems as a Vehicle to Compare Efficiencies of Neural Network Architectures , 2011 .

[22]  Wei He,et al.  Adaptive Fuzzy Neural Network Control for a Constrained Robot Using Impedance Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Ricardo Nicolau Nassar Koury,et al.  Prediction of wind speed and wind direction using artificial neural network, support vector regression and adaptive neuro-fuzzy inference system , 2018 .

[24]  Hao Yu,et al.  Neural Network Learning Without Backpropagation , 2010, IEEE Transactions on Neural Networks.

[25]  Junfei Qiao,et al.  Identification and modeling of nonlinear dynamical systems using a novel self-organizing RBF-based approach , 2012, Autom..

[26]  Jyoti Yadav,et al.  Classification of EEG signals for epileptic seizures using Levenberg-Marquardt algorithm based Multilayer Perceptron Neural Network , 2018, J. Intell. Fuzzy Syst..

[27]  P. Rozycki,et al.  Dedicated deep neural network architectures and methods for their training , 2015, 2015 IEEE 19th International Conference on Intelligent Engineering Systems (INES).

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  My Abdelouahed Sabri,et al.  Traffic flow prediction using neural network , 2018, 2018 International Conference on Intelligent Systems and Computer Vision (ISCV).

[30]  Bing Chen,et al.  Neural Network-Based Adaptive Dynamic Surface Control for Permanent Magnet Synchronous Motors , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Bo Wu,et al.  Neural Network Training With Levenberg–Marquardt and Adaptable Weight Compression , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Janusz Kolbusz,et al.  The Study of Architecture MLP with Linear Neurons in Order to Eliminate the "vanishing Gradient" Problem , 2017, ICAISC.

[33]  Janusz Kolbusz,et al.  Implementation of Deep Neural Networks for Industry Applications , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[34]  David A. Wagner,et al.  Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples , 2018, ICML.

[35]  Hao Yu,et al.  An Incremental Design of Radial Basis Function Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.