A time-dependent enhanced support vector machine for time series regression

Support Vector Machines (SVMs) are a leading tool in machine learning and have been used with considerable success for the task of time series forecasting. However, a key challenge when using SVMs for time series is the question of how to deeply integrate time elements into the learning process. To address this challenge, we investigated the distribution of errors in the forecasts delivered by standard SVMs. Once we identified the samples that produced the largest errors, we observed their correlation with distribution shifts that occur in the time series. This motivated us to propose a time-dependent loss function which allows the inclusion of the information about the distribution shifts in the series directly into the SVM learning process. We present experimental results which indicate that using a time-dependent loss function is highly promising, reducing the overall variance of the errors, as well as delivering more accurate predictions.

[1]  Alexander J. Smola,et al.  Bundle Methods for Regularized Risk Minimization , 2010, J. Mach. Learn. Res..

[2]  Lei Xu,et al.  Independent component ordering in ICA time series analysis , 2001, Neurocomputing.

[3]  Eamonn J. Keogh,et al.  Scaling and time warping in time series querying , 2005, The VLDB Journal.

[4]  Saso Dzeroski,et al.  Analysis of Time Series Data with Predictive Clustering Trees , 2006, KDID.

[5]  Xindong Wu,et al.  Mining distribution change in stock order streams , 2010, 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010).

[6]  Lars Schmidt-Thieme,et al.  Factorizing Markov Models for Categorical Time Series Prediction , 2011 .

[7]  Masashi Sugiyama,et al.  Change-Point Detection in Time-Series Data by Direct Density-Ratio Estimation , 2009, SDM.

[8]  Nguyen Lu Dang Khoa,et al.  Robust Outlier Detection Using Commute Time and Eigenspace Embedding , 2010, PAKDD.

[9]  Ming-Wei Chang,et al.  Analysis of Nonstationary Time Series Using Support Vector Machines , 2002, SVM.

[10]  Tim Oates,et al.  Visualizing Variable-Length Time Series Motifs , 2012, SDM.

[11]  Varun Chandola,et al.  A Gaussian Process Based Online Change Detection Algorithm for Monitoring Periodic Time Series , 2011, SDM.

[12]  Ming-Syan Chen,et al.  Adaptive Clustering for Multiple Evolving Streams , 2006, IEEE Transactions on Knowledge and Data Engineering.

[13]  Ee-Peng Lim,et al.  Analyzing feature trajectories for event detection , 2007, SIGIR.

[14]  Pang-Ning Tan,et al.  An Integrated Framework for Simultaneous Classification and Regression of Time-Series Data , 2010, SDM.

[15]  Sanjay Chawla,et al.  A Quadratic Mean based Supervised Learning Model for Managing Data Skewness , 2011, SDM.

[16]  Sergio Greco,et al.  Effective and efficient similarity search in time series , 2006, CIKM '06.

[17]  Robert Fildes,et al.  Journal of forecasting 7: Robert F. Engle, Scott J. Brown and Gary Stern, “A comparison of adaptive structural forecasting methods for electricity sales”, (1988) 149–172 , 1989 .

[18]  Yan Liu,et al.  Sparse-GEV: Sparse Latent Space Model for Multivariate Extreme Value Time Serie Modeling , 2012, ICML.

[19]  G. C. Tiao,et al.  Random Level-Shift Time Series Models, ARIMA Approximations, and Level-Shift Detection , 1990 .

[20]  Kun-Huang Huarng,et al.  The application of neural networks to forecast fuzzy time series , 2006 .

[21]  Tzung-Pei Hong,et al.  Cluster-based genetic segmentation of time series with DWT , 2009, Pattern Recognit. Lett..

[22]  Zoran Obradovic,et al.  Rapid design of neural networks for time series prediction , 1996 .

[23]  Philip S. Yu,et al.  Extracting Interpretable Features for Early Classification on Time Series , 2011, SDM.

[24]  Valentina Corradi,et al.  Predicting the volatility of the S&P-500 stock index via GARCH models: the role of asymmetries , 2005 .

[25]  Chih-Ling Tsai,et al.  Outlier Detections in Autoregressive Models , 2003 .

[26]  R. Tsay.  Outliers, Level Shifts, and Variance Changes in Time Series , 1988 .

[27]  Pang-Ning Tan,et al.  Detection and Characterization of Anomalies in Multivariate Time Series , 2009, SDM.

[28]  Lai-Wan Chan,et al.  Support Vector Machine Regression for Volatile Stock Market Prediction , 2002, IDEAL.

[29]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.

[30]  Eamonn J. Keogh,et al.  Online discovery and maintenance of time series motifs , 2010, KDD.