A least third-order cumulants objective function

A novel Least Cumulants Method is proposed to tackle the problem of fitting to underlying function in small data sets with high noise level because higher-order statistics provide an unique feature of suppressing Gaussian noise processes of unknown spectral characteristics. The current backpropagation algorithm is actually the Least Square Method based algorithm which does not perform very well in noisy data set. Instead, the proposed method is more robust to the noise because a complete new objective function based on higher-order statistics is introduced. The proposed objective function was validated by applying to predict benchmark sunspot data and excellent results are obtained. The proposed objective function enables the network to provide a very low training error and excellent generalization property. Our results indicate that the network trained by the proposed objective function can, at most, provide 73% reduction of normalized test error in the benchmark test.

[1]  Jerry M. Mendel,et al.  Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications , 1991, Proc. IEEE.

[2]  Chrysostomos L. Nikias,et al.  Higher-order spectral analysis , 1993, Proceedings of the 15th Annual International Conference of the IEEE Engineering in Medicine and Biology Societ.

[3]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[4]  A. V.DavidSánchez,et al.  Robustization of a learning method for RBF networks , 1995, Neurocomputing.

[5]  S. Ergezinger,et al.  An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer , 1995, IEEE Trans. Neural Networks.

[6]  Tommy W. S. Chow,et al.  Accelerated training algorithm for feedforward neural networks based on least squares method , 2005, Neural Processing Letters.

[7]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[8]  Lars Kai Hansen,et al.  On design and evaluation of tapped-delay neural network architectures , 1993, IEEE International Conference on Neural Networks.

[9]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[10]  Gustavo Deco,et al.  Unsupervised Mutual Information Criterion for Elimination of Overtraining in Supervised Multilayer Networks , 1995, Neural Computation.

[11]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[12]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[13]  Ramesh C. Jain,et al.  A robust backpropagation learning algorithm for function approximation , 1994, IEEE Trans. Neural Networks.

[14]  M.R. Raghuveer,et al.  Bispectrum estimation: A digital signal processing framework , 1987, Proceedings of the IEEE.

[15]  Georgios B. Giannakis,et al.  A unifying maximum-likelihood view of cumulant and polyspectral measures for non-Gaussian signal classification and estimation , 1992, IEEE Trans. Inf. Theory.