A novel noise robust fourth-order cumulants cost function

Abstract A novel robust fourth-order cumulants cost function was introduced to enhance the fitting to an underlying function in small data sets with high noise level of Gaussian distribution because higher-order statistics provide a unique feature of suppressing Gaussian noise processes of unknown spectral characteristics. The proposed cost function was validated on the prediction of benchmark sunspot data and an excellent result was obtained. The proposed cost function enables the network to provide a very low training error and an excellent generalization property. Our result indicates that the network trained by the proposed cost function can, at most, provide 74% reduction of the normalized test error in the benchmark test.

[1]  M. Bertero.  Regularization methods for linear inverse problems , 1986 .

[2]  Gustavo Deco,et al.  Unsupervised Mutual Information Criterion for Elimination of Overtraining in Supervised Multilayer Networks , 1995, Neural Computation.

[3]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[4]  Mahmood R. Azimi-Sadjadi,et al.  Fast learning process of multilayer neural networks using recursive least squares method , 1992, IEEE Trans. Signal Process..

[5]  S. Ergezinger,et al.  An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer , 1995, IEEE Trans. Neural Networks.

[6]  Lars Kai Hansen,et al.  On design and evaluation of tapped-delay neural network architectures , 1993, IEEE International Conference on Neural Networks.

[7]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[8]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[9]  Grace Wahba.  Smoothing and Ill-Posed Problems , 1979 .

[10]  T. Chow,et al.  Nonlinear autoregressive integrated neural network model for short-term load forecasting , 1996 .

[11]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[12]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[13]  Jerry M. Mendel,et al.  Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications , 1991, Proc. IEEE.

[14]  Frank Bärmann,et al.  A learning algorithm for multilayered neural networks based on linear least squares problems , 1993, Neural Networks.

[15]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[16]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[17]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[18]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[19]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.