Online monitoring and prediction of complex time series events from nonstationary time series data

OF THE DISSERTATION Online Monitoring and Prediction of Complex Time Series Events From Nonstationary Time Series Data by Shouyi Wang Dissertation Director: Wanpracha Art Chaovalitwongse Much of the world’s supply of data is in the form of time series. In the last decade, there has been an explosion of interest in time series data mining. Time series prediction has been widely used in engineering, economy, industrial manufacturing, finance, management and many other fields. Many new algorithms have been developed to classify, cluster, segment, index, discover rules, and detect anomalies/novelties in time series. However, traditional time series analysis methods are limited by the requirement of stationarity of the time series and normality and independence of the residuals. Because they attempt to characterize and predict all time series observations, traditional time series analysis methods are unable to identify complex (nonperiodic, nonlinear, irregular, and chaotic) characteristics. As a result, the prediction of multivariate noisy time series (such as physiological signals) is still very challenging due to high noise, non-stationarity, and non-linearity. The objective of this research is to develop new reliable frameworks for analyzing multivariate noisy time series, and to apply the framework to online monitor noisy time series and predict critical events online. In particular, this research made an extensive study on one important form of multivariate time series: electroencephalography (EEG) data, based on which two new online monitoring and prediction frameworks for multivariate time series were introduced and evaluated. The new online monitoring and

[1]  C. Elger,et al.  CAN EPILEPTIC SEIZURES BE PREDICTED? EVIDENCE FROM NONLINEAR TIME SERIES ANALYSIS OF BRAIN ELECTRICAL ACTIVITY , 1998 .

[2]  Ivan Osorio,et al.  Strategies for adapting automated seizure detection algorithms. , 2007, Medical engineering & physics.

[3]  Kai-Quan Shen,et al.  EEG-based mental fatigue measurement using multi-class support vector machines with confidence estimate , 2008, Clinical Neurophysiology.

[4]  B. Bowerman,et al.  Forecasting and Time Series: An Applied Approach , 2000 .

[5]  Eytan Domany,et al.  Coupled Two-way Clustering Analysis of Breast Cancer and Colon Cancer Gene Expression Data , 2002, Bioinform..

[6]  Martti Juhola,et al.  Syntactic recognition of ECG signals by attributed finite automata , 1995, Pattern Recognit..

[7]  E. S. Gardner EXPONENTIAL SMOOTHING: THE STATE OF THE ART, PART II , 2006 .

[8]  P. Sajda,et al.  Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[10]  Charles W. Anderson,et al.  Classification of EEG Signals from Four Subjects During Five Mental Tasks , 2007 .

[11]  Erkki Oja,et al.  Artificial Neural Networks: Biological Inspirations - ICANN 2005, 15th International Conference, Warsaw, Poland, September 11-15, 2005, Proceedings, Part I , 2005, ICANN.

[12]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[13]  Wesley W. Chu,et al.  Segment-based approach for subsequence searches in sequence databases , 2001, Comput. Syst. Sci. Eng..

[14]  Klaus Lehnertz,et al.  Automated detection of a preseizure state based on a decrease in synchronization in intracranial electroencephalogram recordings from epilepsy patients. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[16]  Deng-Shan Shiau,et al.  Predictability Analysis for an Automated Seizure Prediction Algorithm , 2006, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[17]  K. R. Ridderinkhof,et al.  Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. , 2001, Psychophysiology.

[18]  Todd C. Handy,et al.  Event-related potentials : a methods handbook , 2005 .

[19]  F. Mormann,et al.  Nonlinear EEG analysis in epilepsy: its possible use for interictal focus localization, seizure anticipation, and prevention. , 2001, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[20]  E. J. Kostelich,et al.  Comparison of Algorithms for Determining Lyapunov Exponents from Experimental Data , 1986 .

[21]  I J Rampil,et al.  No correlation between quantitative electroencephalographic measurements and movement response to noxious stimuli during isoflurane anesthesia in rats. , 1992, Anesthesiology.

[22]  O. A. Rosso,et al.  EEG analysis using wavelet-based information tools , 2006, Journal of Neuroscience Methods.

[23]  Klaus-Robert Müller,et al.  Subject-independent mental state classification in single trials , 2009, Neural Networks.

[24]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[25]  A. Schulze-Bonhage,et al.  Joining the benefits: Combining epileptic seizure prediction methods , 2010, Epilepsia.

[26]  D. Peel,et al.  Economic Forecasting: An Introduction. , 1992 .

[27]  Jun Morimoto,et al.  Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[28]  Michael S. Ford,et al.  The Illustrated Wavelet Transform Handbook: Introductory Theory and Applications in Science , 2003 .

[29]  A. Kraskov,et al.  On the predictability of epileptic seizures , 2005, Clinical Neurophysiology.

[30]  Andrew Britton Gardner,et al.  A Novelty Detection Approach to Seizure Analysis from Intracranial EEG , 2004 .

[31]  T. Tomson,et al.  Risk of Extremity Fractures in Adult Outpatients with Epilepsy , 2002, Epilepsia.

[32]  William J. Williams,et al.  Nonlinear Dynamics of Electrocorticographic Data in Temporal Lobe Epilepsy , 1988 .

[33]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[34]  G. Vachtsevanos,et al.  Epileptic Seizures May Begin Hours in Advance of Clinical Onset A Report of Five Patients , 2001, Neuron.

[35]  Zhaohong Deng,et al.  Clustering-Inverse: A Generalized Model for Pattern-Based Time Series Segmentation , 2011, J. Intell. Learn. Syst. Appl..

[36]  Timo Koskela,et al.  Neural network methods in analysing and modelling time varying processes , 2003 .

[37]  Jürgen Schmidhuber,et al.  A reinforcement learning approach for individualizing erythropoietin dosages in hemodialysis patients , 2009, Expert Syst. Appl..

[38]  Maria E. Orlowska,et al.  Finding Temporal Features of Event-Oriented Patterns , 2005, PAKDD.

[39]  M. Ringnér,et al.  Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks , 2001, Nature Medicine.

[40]  Dinggang Shen,et al.  Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection , 2005, NeuroImage.

[41]  Anthony L Bertapelle Spectral Analysis of Time Series. , 1979 .

[42]  Robert M. Worth,et al.  Real-time seizure prediction from local field potentials using an adaptive Wiener algorithm , 2010, Comput. Biol. Medicine.

[43]  Alain Rakotomamonjy,et al.  Ensemble of SVMs for Improving Brain Computer Interface P300 Speller Performances , 2005, ICANN.

[44]  Leonidas D. Iasemidis,et al.  On the dynamics of the human brain in temporal lobe epilepsy. , 1991 .

[45]  Viglione Ss,et al.  Proceedings: Epileptic seizure prediction. , 1975 .

[46]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[47]  Donald Ervin Knuth,et al.  The Art of Computer Programming, Volume II: Seminumerical Algorithms , 1970 .

[48]  Shabbar F. Danish,et al.  Functional localization and visualization of the subthalamic nucleus from microelectrode recordings acquired during DBS surgery with unsupervised machine learning , 2009, Journal of neural engineering.

[49]  Wanpracha Art Chaovalitwongse,et al.  Evaluating and Comparing Forecasting Models , 2011 .

[50]  Richard J. Povinelli,et al.  A New Temporal Pattern Identification Method for Characterization and Prediction of Complex Time Series Events , 2003, IEEE Trans. Knowl. Data Eng..

[51]  Bernhard Sick,et al.  Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Michael L. Anderson,et al.  Brain Network Analysis of Seizure Evolution , 2008 .

[53]  S. Huffel,et al.  Anticipation of epileptic seizures from standard EEG recordings , 2003, The Lancet.

[54]  W. J. Williams,et al.  Modelling of ECoG in temporal lobe epilepsy. , 1988, Biomedical sciences instrumentation.

[55]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[56]  Yili Liu,et al.  Queuing Network Modeling of Transcription Typing , 2008, TCHI.

[57]  P. Pardalos,et al.  Performance of a seizure warning algorithm based on the dynamics of intracranial EEG , 2005, Epilepsy Research.

[58]  Eugene Fink,et al.  Search for Patterns in Compressed Time Series , 2002, Int. J. Image Graph..

[59]  Hong Tang,et al.  Data mining techniques for cancer detection using serum proteomic profiling , 2004, Artif. Intell. Medicine.

[60]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[61]  David R. Cox,et al.  Time Series Analysis , 2012 .

[62]  B. G. Quinn,et al.  The determination of the order of an autoregression , 1979 .

[63]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[64]  Everette S. Gardner,et al.  Exponential smoothing: The state of the art , 1985 .

[65]  J P Lieb,et al.  Temporo-spatial patterns of pre-ictal spike activity in human temporal lobe epilepsy. , 1983, Electroencephalography and clinical neurophysiology.

[66]  Shivkumar Sabesan,et al.  Nonlinear dynamics of seizure prediction in a rodent model of epilepsy. , 2010, Nonlinear dynamics, psychology, and life sciences.

[67]  Brian Litt,et al.  The statistics of a practical seizure warning system , 2008, Journal of neural engineering.

[68]  Anthony Ralston,et al.  Mathematical Methods for Digital Computers, Volume II. , 1968 .

[69]  J. Martinerie,et al.  Preictal state identification by synchronization changes in long-term intracranial EEG recordings , 2005, Clinical Neurophysiology.

[70]  Charles W. Anderson,et al.  Discriminating mental tasks using EEG represented by AR models , 1995, Proceedings of 17th International Conference of the Engineering in Medicine and Biology Society.

[71]  Hao Feng,et al.  Online signature verification using a new extreme points warping technique , 2003, Pattern Recognit. Lett..

[72]  A. Schulze-Bonhage,et al.  How well can epileptic seizures be predicted? An evaluation of a nonlinear method. , 2003, Brain : a journal of neurology.

[73]  N C Andreasen,et al.  Effects of errors in a multicenter medical study: preventing misinterpreted data. , 1994, Journal of psychiatric research.

[74]  Jean Gotman,et al.  Adaptive segmentation of electroencephalographic data using a nonlinear energy operator , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[75]  Pavel Senin,et al.  Dynamic Time Warping Algorithm Review , 2008 .

[76]  Maria E. Orlowska,et al.  Finding Event-Oriented Patterns in Long Temporal Sequences , 2003, PAKDD.

[77]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter. , 1991 .

[78]  B. Malakooti Forecasting , 2013 .

[79]  J. F. Kaiser,et al.  On a simple algorithm to calculate the 'energy' of a signal , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[80]  V. Srinivasan,et al.  Approximate Entropy-Based Epileptic EEG Detection Using Artificial Neural Networks , 2007, IEEE Transactions on Information Technology in Biomedicine.

[81]  A. Kanemitsu,et al.  [Anatomy of the brain]. , 1987, Nihon rinsho. Japanese journal of clinical medicine.

[82]  Henry D. I. Abarbanel,et al.  Analysis of Observed Chaotic Data , 1995 .

[83]  Ya-Ju Fan,et al.  Novel Optimization Models for Abnormal Brain Activity Classification , 2008, Oper. Res..

[84]  Donald J. Berndt,et al.  Finding Patterns in Time Series: A Dynamic Programming Approach , 1996, Advances in Knowledge Discovery and Data Mining.

[85]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[86]  Brian Litt,et al.  Line length: an efficient feature for seizure onset detection , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[87]  Eugene Fink,et al.  Indexing of time series by major minima and maxima , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[88]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[89]  Chng Eng Siong,et al.  High accuracy classification of EEG signal , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[90]  I. Osorio,et al.  Real‐Time Automated Detection and Quantitative Analysis of Seizures and Short‐Term Prediction of Clinical Onset , 1998, Epilepsia.

[91]  R. Quiroga,et al.  Kulback-Leibler and renormalized entropies: applications to electroencephalograms of epilepsy patients. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[92]  Jim Hunter,et al.  Knowledge-Based Event Detection in Complex Time Series Data , 1999, AIMDM.

[93]  Alexandre Andrade,et al.  Correlation Dimension Maps of EEG from Epileptic Absences , 1999, Brain Topography.

[94]  Sander Scholtus Automatic correction of simple typing errors in numerical data with balance edits , 2009 .

[95]  Changxu Wu,et al.  Detecting Typing Errors in a Numerical Typing Task with Linear Discriminant Analysis of Single Trial EEG , 2009 .

[96]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[97]  Ada Wai-Chee Fu,et al.  Efficient time series matching by wavelets , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[98]  S S Viglione,et al.  Proceedings: Epileptic seizure prediction. , 1975, Electroencephalography and clinical neurophysiology.

[99]  Alistair I. Mees,et al.  Dynamics of brain electrical activity , 2005, Brain Topography.

[100]  W. Art Chaovalitwongse,et al.  Dynamical approaches and multi-quadratic integer programming for seizure prediction , 2005, Optim. Methods Softw..

[101]  Shari Trewin An invisible keyguard , 2002, Assets '02.

[102]  Eamonn Keogh A Fast and Robust Method for Pattern Matching in Time Series Databases , 2012 .

[103]  Theodosios Pavlidis,et al.  Segmentation of Plane Curves , 1974, IEEE Transactions on Computers.

[104]  Yoonkyung Lee,et al.  Classification of Multiple Cancer Types by Multicategory Support Vector Machines Using Gene Expression Data , 2003, Bioinform..

[105]  David D. Jensen,et al.  Mining of Concurrent Text and Time Series , 2008 .

[106]  R. Esteller,et al.  Comparison of line length feature before and after brain electrical stimulation in epileptic patients , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[107]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[108]  Chonghui Guo,et al.  Similarity measure based on piecewise linear approximation and derivative dynamic time warping for time series mining , 2011, Expert Syst. Appl..

[109]  Richard L. Smith,et al.  PREDICTIVE INFERENCE , 2004 .

[110]  Eamonn J. Keogh,et al.  An Enhanced Representation of Time Series Which Allows Fast and Accurate Classification, Clustering and Relevance Feedback , 1998, KDD.

[111]  Yasuo Ohashi,et al.  A comparison of error detection rates between the reading aloud method and the double data entry method. , 2003, Controlled clinical trials.

[112]  G.A. Barreto,et al.  On the classification of mental tasks: a performance comparison of neural and statistical approaches , 2004, Proceedings of the 2004 14th IEEE Signal Processing Society Workshop Machine Learning for Signal Processing, 2004..

[113]  Eamonn J. Keogh,et al.  Segmenting Time Series: A Survey and Novel Approach , 2002 .

[114]  Keinosuke Fukunaga,et al.  Statistical Pattern Recognition , 1993, Handbook of Pattern Recognition and Computer Vision.

[115]  I. Gath,et al.  Prediction of epileptic seizures from two-channel EEG , 1998, Medical and Biological Engineering and Computing.

[116]  Sebastian Mika,et al.  Kernel Fisher Discriminants , 2003 .

[117]  E. Hannan The Estimation of the Order of an ARMA Process , 1980 .

[118]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[119]  F. Mormann,et al.  Seizure prediction: the long and winding road. , 2007, Brain : a journal of neurology.

[120]  Luigi Chisci,et al.  Real-Time Epileptic Seizure Prediction Using AR Models and Support Vector Machines , 2010, IEEE Transactions on Biomedical Engineering.

[121]  Panos M. Pardalos,et al.  On the time series support vector machine using dynamic time warping kernel for brain activity classification , 2008 .

[122]  Hagai Bergman,et al.  Real‐time refinement of subthalamic nucleus targeting using Bayesian decision‐making on the root mean square measure , 2006, Movement disorders : official journal of the Movement Disorder Society.

[123]  Ya-Ju Fan,et al.  On the Time Series $K$-Nearest Neighbor Classification of Abnormal Brain Activity , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[124]  Bruce L. Bowerman,et al.  Forecasting and time series: An applied approach. 3rd. ed. , 1993 .

[125]  T. Salthouse Perceptual, cognitive, and motoric aspects of transcription typing. , 1986, Psychological bulletin.

[126]  D. Rubinfeld,et al.  Econometric models and economic forecasts , 2002 .

[127]  Daniel Lemire,et al.  A Better Alternative to Piecewise Linear Time Series Segmentation , 2006, SDM.

[128]  S. Huffel,et al.  Anticipation of epileptic seizures from standard EEG recordings , 2003, The Lancet.

[129]  Helge J. Ritter,et al.  BCI competition 2003-data set IIb: support vector machines for the P300 speller paradigm , 2004, IEEE Transactions on Biomedical Engineering.

[130]  M Sachs [Anatomy of the brain]. , 1982, Soins; la revue de reference infirmiere.

[131]  J. Shukla,et al.  Predictability in the midst of chaos: A scientific basis for climate forecasting , 1998, Science.

[132]  Jin Yu,et al.  Natural Actor-Critic for Road Traffic Optimisation , 2006, NIPS.

[133]  G. Weiss Aspects and Applications of the Random Walk , 1994 .

[134]  Huaiqing Wang,et al.  Novel Online Methods for Time Series Segmentation , 2008, IEEE Transactions on Knowledge and Data Engineering.

[135]  David M. Young Data Inaccuracy in the Global Transportation Network , 2012 .

[136]  R. Acharya U,et al.  Nonlinear analysis of EEG signals at different mental states , 2004, Biomedical engineering online.

[137]  Eamonn J. Keogh,et al.  A Probabilistic Approach to Fast Pattern Matching in Time Series Databases , 1997, KDD.

[138]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[139]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[140]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[141]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[142]  Abdulhamit Subasi,et al.  EEG signal classification using wavelet feature extraction and a mixture of expert model , 2007, Expert Syst. Appl..

[143]  H. Akaike A new look at the statistical model identification , 1974 .

[144]  W. Art Chaovalitwongse,et al.  Adaptive epileptic seizure prediction system , 2003, IEEE Transactions on Biomedical Engineering.

[145]  Brian Litt,et al.  Epileptic seizure prediction using hybrid feature selection over multiple intracranial EEG electrode contacts: a report of four patients , 2003, IEEE Transactions on Biomedical Engineering.

[146]  Dimitrios Gunopulos,et al.  Streaming Time Series Summarization Using User-Defined Amnesic Functions , 2008, IEEE Transactions on Knowledge and Data Engineering.

[147]  Eamonn J. Keogh,et al.  Locally adaptive dimensionality reduction for indexing large time series databases , 2001, SIGMOD '01.

[148]  C. Siriopoulos,et al.  Time series forecasting with a hybrid clustering scheme and pattern recognition , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[149]  Hagit Shatkay,et al.  Approximate queries and representations for large data sequences , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[150]  K. Lehnertz,et al.  The First International Collaborative Workshop on Seizure Prediction: summary and data description , 2005, Clinical Neurophysiology.

[151]  Kemal Polat,et al.  Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform , 2007, Appl. Math. Comput..

[152]  Andrew Emili,et al.  Integrating gene and protein expression data: pattern analysis and profile mining. , 2005, Methods.

[153]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[154]  Dimitrios I. Fotiadis,et al.  EEG Transient Event Detection and Classification Using Association Rules , 2006, IEEE Transactions on Information Technology in Biomedicine.

[155]  R. T. Pivik,et al.  Guidelines for the recording and quantitative analysis of electroencephalographic activity in research contexts. , 1993, Psychophysiology.

[156]  K. Tsakalis,et al.  Long-term prospective on-line real-time seizure prediction , 2005, Clinical Neurophysiology.

[157]  Shijian Lu,et al.  Subject-independent brain computer interface through boosting , 2008, 2008 19th International Conference on Pattern Recognition.

[158]  Timothy A. Pedley,et al.  Epilepsy : a comprehensive textbook , 2008 .

[159]  Hasan Ocak,et al.  Optimal classification of epileptic seizures in EEG using wavelet analysis and genetic algorithm , 2008, Signal Process..

[160]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[161]  Touradj Ebrahimi,et al.  Support vector EEG classification in the Fourier and time-frequency correlation domains , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[162]  W. Chu,et al.  Fast retrieval of similar subsequences in long sequence databases , 1999, Proceedings 1999 Workshop on Knowledge and Data Engineering Exchange (KDEX'99) (Cat. No.PR00453).

[163]  Peng Wang,et al.  Machine learning in bioinformatics: A brief survey and recommendations for practitioners , 2006, Comput. Biol. Medicine.

[164]  C.W. Anderson,et al.  Comparison of linear, nonlinear, and feature selection methods for EEG signal classification , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[165]  Andrej Pázman,et al.  Nonlinear Regression , 2019, Handbook of Regression Analysis With Applications in R.