A study of singular spectrum analysis with global optimization techniques

Singular spectrum analysis has recently become an attractive tool in a broad range of applications. Its main mechanism of alternating between rank reduction and Hankel projection to produce an approximation to a particular component of the original time series, however, deserves further mathematical justification. One paramount question to ask is how good an approximation that such a straightforward apparatus can provide when comparing to the absolute optimal solution. This paper reexamines this issue by exploiting a natural parametrization of a general Hankel matrix via its Vandermonde factorization. Such a formulation makes it possible to recast the notion of singular spectrum analysis as a semi-linear least squares problem over a compact feasible set, whence global optimization techniques can be employed to find the absolute best approximation. This framework might not be immediately suitable for practical application because global optimization is expectedly more expensive, but it does provide a theoretical baseline for comparison. As such, our empirical results indicate that the simpler SSA algorithm usually is amazingly sufficient as a handy tool for constructing exploratory model. The more complicated global methods could be used as an alternative of rigorous affirmative procedure for verifying or assessing the quality of approximation.

[1]  F. R. Gantmakher The Theory of Matrices , 1984 .

[2]  G. R. Luckhurst,et al.  Director alignment by crossed electric and magnetic fields: a deuterium NMR study. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  V. Moskvina,et al.  Approximate Projectors in Singular Spectrum Analysis , 2002, SIAM J. Matrix Anal. Appl..

[4]  J. B. Rosen,et al.  Low Rank Approximation of a Hankel Matrix by Structured Total Least Norm , 1999 .

[5]  R. Shanmugam Introduction to Time Series and Forecasting , 1997 .

[6]  D. Sornette,et al.  Data-adaptive wavelets and multi-scale singular-spectrum analysis , 1998, chao-dyn/9810034.

[7]  Dan Kalman,et al.  The Generalized Vandermonde Matrix , 1984 .

[8]  Dimitrios D. Thomakos,et al.  A review on singular spectrum analysis for economic and financial time series , 2010 .

[9]  Daniel Boley,et al.  Vandermonde Factorization of a Hankel Matrix ? , 2006 .

[10]  David Popivanov,et al.  Method for single-trial readiness potential identification, based on singular spectrum analysis , 1996, Journal of Neuroscience Methods.

[11]  V. Moskvina,et al.  An Algorithm Based on Singular Spectrum Analysis for Change-Point Detection , 2003 .

[12]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[13]  A. Zhigljavsky,et al.  Analysis of time series structure , 2013 .

[14]  Klaus Fraedrich,et al.  Estimating the Dimensions of Weather and Climate Attractors , 1986 .

[15]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[16]  David A. Wismer,et al.  Introduction to nonlinear optimization , 1978 .

[17]  Kenneth S. Miller,et al.  Complex Linear Least Squares , 1973 .

[18]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[19]  A. Zhigljavsky Stochastic Global Optimization , 2008, International Encyclopedia of Statistical Science.

[20]  Peter Salamon,et al.  Facts, Conjectures, and Improvements for Simulated Annealing , 1987 .

[21]  Michael D. Vose,et al.  The simple genetic algorithm - foundations and theory , 1999, Complex adaptive systems.

[22]  Georg Heinig,et al.  Vandermonde factorization and canonical representations of block hankel matrices , 1996 .

[23]  Charles Audet,et al.  Globalization strategies for Mesh Adaptive Direct Search , 2008, Comput. Optim. Appl..

[24]  J. Elsner,et al.  Singular Spectrum Analysis: A New Tool in Time Series Analysis , 1996 .

[25]  Anatoly Zhigljavsky,et al.  Multivariate singular spectrum analysis for forecasting revisions to real-time data , 2011 .

[26]  Anatoly Zhigljavsky,et al.  Predicting daily exchange rate with singular spectrum analysis , 2010 .

[27]  R. Vautard,et al.  Singular-spectrum analysis: a toolkit for short, noisy chaotic signals , 1992 .

[28]  Hossein Hassani,et al.  MULTIVARIATE SINGULAR SPECTRUM ANALYSIS: A GENERAL VIEW AND NEW VECTOR FORECASTING APPROACH , 2013 .

[29]  A. Zhigljavsky,et al.  Forecasting European industrial production with singular spectrum analysis , 2009 .

[30]  D. Arov,et al.  Infinite Hankel matrices and generalized caretheodory-fejer and I. schur Problems , 1968 .

[31]  Vladimir Peller,et al.  An Excursion into the Theory of Hankel Operators , 1998 .

[32]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[33]  M. Kreĭn,et al.  Infinite hankel matrices and generalized carathéodory — fejer and riesz problems , 1968 .

[34]  Michael Ghil,et al.  Advanced spectral-analysis methods , 1997 .

[35]  J. Elsner Analysis of Time Series Structure: SSA and Related Techniques , 2002 .

[36]  F. Varadi,et al.  Searching for Signal in Noise by Random-Lag Singular Spectrum Analysis , 1999 .

[37]  A. A. Zhigli︠a︡vskiĭ,et al.  Stochastic Global Optimization , 2007 .

[38]  Hossein Hassani,et al.  Singular Spectrum Analysis: Methodology and Comparison , 2021, Journal of Data Science.

[39]  Charles M. Bowden Boundedness of linear operators in the space l2 , 2009 .

[40]  Rahim Mahmoudvand,et al.  On the separability between signal and noise in singular spectrum analysis , 2012 .

[41]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[42]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[43]  W. J. Cunningham,et al.  Introduction to Nonlinear Analysis , 1959 .

[45]  B. Francis,et al.  A Course in H Control Theory , 1987 .

[46]  Z. Nehari On Bounded Bilinear Forms , 1957 .

[47]  Michael Ghil,et al.  Multivariate singular spectrum analysis and the road to phase synchronization. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[49]  M. Auvergne Singular value analysis applied to phase space reconstruction of pulsating stars. , 1988 .

[50]  Rahim Mahmoudvand,et al.  Separability and window length in singular spectrum analysis , 2011 .

[51]  R. Plemmons,et al.  Structured low rank approximation , 2003 .

[52]  S. Power Hankel Operators on Hilbert Space , 1980 .

[53]  Hossein Hassani,et al.  Singular Spectrum Analysis Based on the Minimum Variance Estimator , 2010 .

[54]  I. S. Iokhvidov Hankel and Toeplitz Matrices and Forms: Algebraic Theory , 1982 .

[55]  Anatoly A. Zhigljavsky,et al.  Singular Spectrum Analysis for Time Series , 2013, International Encyclopedia of Statistical Science.

[56]  V. Peller Hankel Operators and Their Applications , 2003, IEEE Transactions on Automatic Control.

[57]  Fred W. Glover,et al.  Scatter Search and Local Nlp Solvers: A Multistart Framework for Global Optimization , 2006, INFORMS J. Comput..

[58]  Boglárka G.-Tóth,et al.  Introduction to Nonlinear and Global Optimization , 2010 .

[59]  Leonard A. Smith,et al.  Monte Carlo SSA: Detecting irregular oscillations in the Presence of Colored Noise , 1996 .

[60]  R. Vautard,et al.  Singular spectrum analysis in nonlinear dynamics, with applications to paleoclimatic time series , 1989 .

[61]  Michael Ghil,et al.  ADVANCED SPECTRAL METHODS FOR CLIMATIC TIME SERIES , 2002 .

[62]  János D. Pintér,et al.  Global optimization in action , 1995 .

[63]  K. Glover All optimal Hankel-norm approximations of linear multivariable systems and their L, ∞ -error bounds† , 1984 .

[64]  M. Vose The Simple Genetic Algorithm , 1999 .

[65]  Christine M. Anderson-Cook Practical Genetic Algorithms (2nd ed.) , 2005 .

[66]  G. P. King,et al.  Extracting qualitative dynamics from experimental data , 1986 .