Noise robustness enhancement using fourth-order cumulants cost function

A novel robust fourth-order cumulants cost function is introduced to enhance the fitting to underlying function in small data sets with high noise level of Gaussian noise. The neural network learns based on the gradient descent optimization method by introducing a constraint term in the cost function. The proposed cost function was applied to benchmark sunspot series prediction and nonlinear system identification. Excellent results are obtained. The neural network can provide lower training error and excellent generalization property. Our proposed cost function enables the network to provide, at most, 73% reduction of normalized test error in the benchmark test.

[1]  Mahmood R. Azimi-Sadjadi,et al.  Fast learning process of multilayer neural networks using recursive least squares method , 1992, IEEE Trans. Signal Process..

[2]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[3]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[4]  Georgios B. Giannakis,et al.  A unifying maximum-likelihood view of cumulant and polyspectral measures for non-Gaussian signal classification and estimation , 1992, IEEE Trans. Inf. Theory.

[5]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[6]  Brian M. Sadler,et al.  Estimation and detection in non-Gaussian noise using higher order statistics , 1994, IEEE Trans. Signal Process..

[7]  Jerry M. Mendel,et al.  Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications , 1991, Proc. IEEE.

[8]  Frank Bärmann,et al.  A learning algorithm for multilayered neural networks based on linear least squares problems , 1993, Neural Networks.

[9]  M.R. Raghuveer,et al.  Bispectrum estimation: A digital signal processing framework , 1987, Proceedings of the IEEE.

[10]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[11]  S. Ergezinger,et al.  An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer , 1995, IEEE Trans. Neural Networks.

[12]  Lars Kai Hansen,et al.  On design and evaluation of tapped-delay neural network architectures , 1993, IEEE International Conference on Neural Networks.

[13]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[14]  Gustavo Deco,et al.  Unsupervised Mutual Information Criterion for Elimination of Overtraining in Supervised Multilayer Networks , 1995, Neural Computation.