Tikhonov-type regularization in local model for noisy chaotic time series prediction

Tikhonov-type regularization method for noisy chaotic time series prediction is investigated. The current regularized local prediction method is interpreted as one kind of filter factors to decrease the variance of the predictor. One drawback in the interpretation is the ignorance of the random noise in coefficient matrix, another drawback is the relationship between the regularization parameter and the noise condition is not clearly explained, so the determination of regularization parameter has to resort to some techniques such as cross validation. In this study, local linear model is studied from the perceptive of the errors-in-variables (EIV) modeling, and the predictor is designed by considering the noise both in coefficient matrix and right-hand side. The optimal solution can be obtained by second order convex program (SOCP) if given a perturbation bound of the noise, and the solution can be reformulated as a form of Tikhonov regularization, and it will be shown how regularization parameter is related to the Frobenius norm of the noise containing in coefficient matrix and right-hand side. Two demonstrations are presented to show the validity of the results.

[1]  Masashi Sugiyama,et al.  Subspace Information Criterion for Model Selection , 2001, Neural Computation.

[2]  S. Chandrasekaran,et al.  Parameter estimation in the presence of bounded modeling errors , 1997, IEEE Signal Processing Letters.

[3]  G. Alistair Watson.  Robust counterparts of errors-in-variables problems , 2007, Comput. Stat. Data Anal..

[4]  Akira Kawamura,et al.  Prediction of unspots using reconstructed chaotic system equations , 1995 .

[5]  José Carlos Príncipe,et al.  Dynamic Modelling of Chaotic Time Series with Neural Networks , 1994, NIPS.

[6]  H. Kantz,et al.  Improved cost functions for modelling of noisy chaotic time series , 1997 .

[7]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[8]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[9]  Kevin Judd,et al.  Refinements to Model Selection for Nonlinear Time Series , 2003, Int. J. Bifurc. Chaos.

[10]  Min Han,et al.  Prediction of chaotic time series based on the recurrent predictor neural network , 2004, IEEE Transactions on Signal Processing.

[11]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[12]  O. Lingjærde,et al.  Regularized local linear prediction of chaotic time series , 1998 .

[13]  C. Rasmussen,et al.  Gaussian Process Priors with Uncertain Inputs - Application to Multiple-Step Ahead Time Series Forecasting , 2002, NIPS.

[14]  Simon Haykin,et al.  Making sense of a complex world , 1998 .

[15]  Jose C. Principe,et al.  Prediction of Chaotic Time Series with Neural Networks , 1992 .

[16]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[17]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[18]  Robert M. Farber,et al.  How Neural Nets Work , 1987, NIPS.

[19]  Diana Maria Sima,et al.  Regularization Techniques in Model Fitting and Parameter Estimation (Regularisatietechnieken in modellering en parameterschatting) , 2006 .

[20]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[21]  S. Haykin,et al.  Making sense of a complex world [chaotic events modeling] , 1998, IEEE Signal Process. Mag..

[22]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[23]  Gene H. Golub,et al.  Regularization by Truncated Total Least Squares , 1997, SIAM J. Sci. Comput..

[24]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[25]  Henry Leung,et al.  Prediction of noisy chaotic time series using an optimal radial basis function neural network , 2001, IEEE Trans. Neural Networks.

[26]  Wei Guo,et al.  Noise Smoothing for Nonlinear Time Series Using Wavelet Soft Threshold , 2007, IEEE Signal Processing Letters.

[27]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[28]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[29]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[30]  Tassos Bountis,et al.  An adaptive way for improving noise reduction using local geometric projection. , 2004, Chaos.

[31]  Eric A. Wan,et al.  Time series prediction by using a connectionist network with internal delay lines , 1993 .