Adaptive Learning-Rate Backpropagation Neural Network Algorithm Based on the Minimization of Mean-Square Deviation for Impulsive Noises

This paper presents a novel adaptive learning-rate backpropagation neural network (ALR-BPNN) algorithm based on the minimization of mean-square deviation (MSD) to implement a fast convergence rate and robustness to impulsive noises. The learning rates of the weights in each hidden layer are derived to minimize the upper bound of the MSD obtained by the analysis, which guarantees a fast convergence rate in a stable range. Moreover, by adopting the variance of the kind of the measurement noises in each layer through the variance of the error signals, the proposed scheme provides robustness to the impulsive noises. The performance of the proposed algorithm is evaluated on various sequential signals and industrial data including the impulsive noise and compared with conventional ALR-BPNN algorithms. Simulation results indicate that the proposed algorithm outperforms the existing algorithms.

[1]  Wei Xing Zheng,et al.  Recursive Adaptive Sparse Exponential Functional Link Neural Network for Nonlinear AEC in Impulsive Noise Environment , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[3]  C. L. Nikias,et al.  Signal processing with fractional lower order moments: stable processes and their applications , 1993, Proc. IEEE.

[4]  Michael K. Weir,et al.  A method for self-determination of adaptive learning rates in back propagation , 1991, Neural Networks.

[5]  Francesco Piazza,et al.  On the complex backpropagation algorithm , 1992, IEEE Trans. Signal Process..

[6]  PooGyeon Park,et al.  Two-stage active noise control with online secondary-path filter based on an adapted scheduled-stepsize NLMS algorithm , 2020, Applied Acoustics.

[7]  Andrew Chi-Sing Leung,et al.  On the Kalman filtering method in neural network training and pruning , 1999, IEEE Trans. Neural Networks.

[8]  PooGyeon Park,et al.  Variable Step-Size Sign Subband Adaptive Filter , 2013, IEEE Signal Processing Letters.

[9]  Lu Xu,et al.  Robust Blind Learning Algorithm for Nonlinear Equalization Using Input Decision Information , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  PooGyeon Park,et al.  A diffusion subband adaptive filtering algorithm for distributed estimation using variable step size and new combination method based on the MSD , 2016, Digit. Signal Process..

[11]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[12]  PooGyeon Park,et al.  A Variable Step-Size Robust Saturation Algorithm Against Impulsive Noises , 2020, IEEE Transactions on Circuits and Systems II: Express Briefs.

[13]  Maciej Niedzwiecki,et al.  Detection of impulsive disturbances in archive audio signals , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  PooGyeon Park,et al.  Variable step-size sign algorithm against impulsive noises , 2015, IET Signal Process..

[15]  Xinghuo Yu,et al.  A general backpropagation algorithm for feedforward neural networks learning , 2002, IEEE Trans. Neural Networks.

[16]  Hao Yu,et al.  Levenberg—Marquardt Training , 2011 .

[17]  Jiashu Zhang,et al.  Adaptively Combined FIR and Functional Link Artificial Neural Network Equalizer for Nonlinear Communication Channel , 2009, IEEE Transactions on Neural Networks.

[18]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[19]  Zheng Bao,et al.  Robust recursive least squares learning algorithm for principal component analysis , 2000, IEEE Trans. Neural Networks Learn. Syst..

[20]  Mohamed Najim,et al.  A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm , 2001, IEEE Trans. Neural Networks.

[21]  Jacek M. Zurada,et al.  A recalling-enhanced recurrent neural network: Conjugate gradient learning algorithm and its convergence analysis , 2020, Inf. Sci..

[22]  Saeid Iranmanesh,et al.  A differential adaptive learning rate method for back-propagation neural networks , 2009 .

[23]  Dilip Sarkar,et al.  Methods to speed up error back-propagation learning algorithm , 1995, CSUR.

[24]  Rudy Setiono,et al.  Use of a quasi-Newton method in a feedforward neural network construction algorithm , 1995, IEEE Trans. Neural Networks.

[25]  David R. Selviah,et al.  Real-Time Nonlinear Parameter Estimation Using the Levenberg–Marquardt Algorithm on Field Programmable Gate Arrays , 2013, IEEE Transactions on Industrial Electronics.

[26]  R. Erb,et al.  Introduction to Backpropagation Neural Network Computation , 1993, Pharmaceutical Research.

[27]  Mark A. Kramer,et al.  Improvement of the backpropagation algorithm for training neural networks , 1990 .

[28]  Ganapati Panda,et al.  Identification of nonlinear dynamic systems using functional link artificial neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[29]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[30]  Samuel Pon Varma Ebenezer Detection of acoustic impulse events in voice applications using a neural network , 2017 .

[31]  Viswanath Ramamurti,et al.  Neural networks for signal detection in non-Gaussian noise , 1997, IEEE Trans. Signal Process..