A new hybrid constructive neural network method for impacting and its application on tungsten price prediction

To accurately predict the price of tungsten with an optimal architecture of neural networks (NNs) and better generalization performance, based on poor generalization and overfitting of a predictor such as a NNs, this paper presents a new hybrid constructive neural network method (HCNNM) to repair the impacting value in the original data in the same manner as the jumping points of a function. A series of theorems was proven that show a function with m jumping discontinuity points (or impacting points) can be approximated with the simplest NNs and a constructive decay Radial basis function (RBF) NNs, and a function with m jumping discontinuity points can be constructively approximated by hybrid constructive NNs. The hybrid networks have an optimal architecture and generalize well. Additionally, a practical problem regarding Tungsten prices from 1900 to 2014 is presented with some impacting points to more accurately approximate the sample data set and forecast future prices with the HCNNM, and some performance measures, such as the training time, testing RMSE and neurons, are compared with traditional algorithms (BP, SVM, ELM and Deep Learning) through many numerical experiments that fully verify the superiority, correctness and validity of the theory.

[1]  Gustavo Deco,et al.  Two Strategies to Avoid Overfitting in Feedforward Networks , 1997, Neural Networks.

[2]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[3]  Skander Soltani,et al.  On the use of the wavelet decomposition for time series prediction , 2002, ESANN.

[4]  Lingling Fan,et al.  Singular Points Detection Based on Zero-Pole Model in Fingerprint Images , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[6]  Gregg D. Wilensky,et al.  Neural Network Studies , 1993 .

[7]  Chien-Jen Huang,et al.  Using multi-stage data mining technique to build forecast model for Taiwan stocks , 2011, Neural Computing and Applications.

[8]  N Mai Duy,et al.  APPROXIMATION OF FUNCTION AND ITS DERIVATIVES USING RADIAL BASIS FUNCTION NETWORKS , 2003 .

[9]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[10]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[11]  C. McGreavy,et al.  Application of wavelets and neural networks to diagnostic system development , 1999 .

[12]  Marco van Akkeren,et al.  A GARCH forecasting model to predict day-ahead electricity prices , 2005, IEEE Transactions on Power Systems.

[13]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[14]  Vince D. Calhoun,et al.  A Parallel Independent Component Analysis Approach to Investigate Genomic Influence on Brain Function , 2008, IEEE Signal Processing Letters.

[15]  MuzhouHou,et al.  A new hybrid constructive neural network method for impacting and its application on tungsten price prediction , 2017 .

[16]  Tao Zhang,et al.  Adaptive neural network control of nonlinear systems by state and output feedback , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[17]  Lei Xu,et al.  Finite Mixture of ARMA-GARCH Model for Stock Price Prediction , 2003 .

[18]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[19]  J. Stock,et al.  A Probability Model of the Coincident Economic Indicators , 1988 .

[20]  Moon Ho Lee,et al.  A new constructive neural network method for noise processing and its application on stock market prediction , 2014, Appl. Soft Comput..

[21]  Kenji Fukumizu,et al.  Relation between weight size and degree of over-fitting in neural network regression , 2008, Neural Networks.

[22]  F. J. Sainz,et al.  Constructive approximate interpolation by neural networks , 2006 .

[23]  Rasmus Berg Palm,et al.  Prediction as a candidate for learning deep hierarchical models of data , 2012 .

[24]  Guangren Duan,et al.  The Design of RBF Neural Networks for Solving Overfitting Problem , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[25]  Shaocheng Tong,et al.  Adaptive Neural Output Feedback Tracking Control for a Class of Uncertain Discrete-Time Nonlinear Systems , 2011, IEEE Transactions on Neural Networks.

[26]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[27]  Yichuang Sun,et al.  Wavelet neural network approach for fault diagnosis of analogue circuits , 2004 .

[28]  Igor V. Tetko,et al.  Neural network studies, 1. Comparison of overfitting and overtraining , 1995, J. Chem. Inf. Comput. Sci..

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[31]  Yi-Fan Wang,et al.  Predicting stock price using fuzzy grey prediction system , 2002, Expert Syst. Appl..

[32]  Liang-Yu Shyu,et al.  Using wavelet transform and fuzzy neural network for VPC detection from the holter ECG , 2004, IEEE Transactions on Biomedical Engineering.

[33]  Meng Joo Er,et al.  Face recognition with radial basis function (RBF) neural networks , 2002, IEEE Trans. Neural Networks.

[34]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[35]  Xuli Han,et al.  Constructive Approximation to Multivariate Function by Decay RBF Neural Network , 2010, IEEE Transactions on Neural Networks.

[36]  Charles K. Chui,et al.  An Introduction to Wavelets , 1992 .

[37]  Hou Muzhou,et al.  A Self-Organizing Mixture Extreme Leaning Machine for Time Series Forecasting * , 2015 .

[38]  Xuehong Zhu,et al.  Decomposition laws of tungsten prices fluctuation since 1900 and its applications , 2013 .

[39]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[40]  Xin Yao,et al.  A New Constructive Algorithm for Architectural and Functional Adaptation of Artificial Neural Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[41]  S. Liong,et al.  GENERALIZATION FOR MULTILAYER NEURAL NETWORK BAYESIAN REGULARIZATION OR EARLY STOPPING , 2004 .

[42]  Hong Wang,et al.  A direct adaptive neural-network control for unknown nonlinear systems and its application , 1998, IEEE Trans. Neural Networks.

[43]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[44]  Francesco Palmieri,et al.  Optimal filtering algorithms for fast learning in feedforward neural networks , 1992, Neural Networks.

[45]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[46]  J. Kynický,et al.  From "strategic" tungsten to "green" neodymium: a century of critical metals at a glance , 2015 .

[47]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[48]  Marijana Zekić.  Neural Network Applications in Stock Market Predictions-A Methodology Analysis , .

[49]  Marios M. Polycarpou,et al.  Stable adaptive neural control scheme for nonlinear systems , 1996, IEEE Trans. Autom. Control..

[50]  Bernard Delyon,et al.  Accuracy analysis for wavelet approximations , 1995, IEEE Trans. Neural Networks.

[51]  Kai Wang,et al.  An Expanded Training Set Based Validation Method to Avoid Overfitting for Neural Network Classifier , 2008, 2008 Fourth International Conference on Natural Computation.

[52]  Derong Liu,et al.  A constructive algorithm for feedforward neural networks with incremental training , 2002 .

[53]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[54]  T.B. Trafalis,et al.  Kernel principal component analysis and support vector machines for stock price prediction , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[55]  J. S. Sahambi,et al.  Classification of ECG arrhythmias using multi-resolution analysis and neural networks , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[56]  J. P. Castagna,et al.  Avoiding overfitting caused by noise using a uniform training mode , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[57]  Raphaël Féraud,et al.  A Fast and Accurate Face Detector Based on Neural Networks , 2001, IEEE Trans. Pattern Anal. Mach. Intell..