Time Series Segmentation through Automatic Feature Learning

Internet of things (IoT) applications have become increasingly popular in recent years, with applications ranging from building energy monitoring to personal health tracking and activity recognition. In order to leverage these data, automatic knowledge extraction - whereby we map from observations to interpretable states and transitions - must be done at scale. As such, we have seen many recent IoT data sets include annotations with a human expert specifying states, recorded as a set of boundaries and associated labels in a data sequence. These data can be used to build automatic labeling algorithms that produce labels as an expert would. Here, we refer to human-specified boundaries as breakpoints. Traditional changepoint detection methods only look for statistically-detectable boundaries that are defined as abrupt variations in the generative parameters of a data sequence. However, we observe that breakpoints occur on more subtle boundaries that are non-trivial to detect with these statistical methods. In this work, we propose a new unsupervised approach, based on deep learning, that outperforms existing techniques and learns the more subtle, breakpoint boundaries with a high accuracy. Through extensive experiments on various real-world data sets - including human-activity sensing data, speech signals, and electroencephalogram (EEG) activity traces - we demonstrate the effectiveness of our algorithm for practical applications. Furthermore, we show that our approach achieves significantly better performance than previous methods.

[1]  Vasilios A. Siris,et al.  Application of anomaly detection algorithms for detecting SYN flooding attacks , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[2]  Chandra Erdman,et al.  A fast Bayesian change point analysis for the segmentation of microarray data , 2008, Bioinform..

[3]  Karsten M. Borgwardt,et al.  Covariate Shift by Kernel Mean Matching , 2009, NIPS 2009.

[4]  Kyong Joo Oh,et al.  Analyzing Stock Market Tick Data Using Piecewise Nonlinear Model , 2022 .

[5]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[6]  Yoshinobu Kawahara,et al.  Change-Point Detection in Time-Series Data Based on Subspace Identification , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[7]  Nigel Collier,et al.  Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation , 2012, Neural Networks.

[8]  Graham J. Williams,et al.  On-Line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms , 2000, KDD '00.

[9]  V. Moskvina,et al.  Application of the singular spectrum analysis for change-point detection in time series , 2006 .

[10]  D. Siegmund,et al.  Sequential multi-sensor change-point detection , 2012, 2013 Information Theory and Applications Workshop (ITA).

[11]  F. Gustafsson The marginalized likelihood ratio test for detecting abrupt changes , 1996, IEEE Trans. Autom. Control..

[12]  J. Bai,et al.  Estimation of a Change Point in Multiple Regression Models , 1997, Review of Economics and Statistics.

[13]  Jaideep Srivastava,et al.  Event detection from time series data , 1999, KDD '99.

[14]  Ruby B. Lee,et al.  Implicit Smartphone User Authentication with Sensors and Contextual Machine Learning , 2017, 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[15]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[16]  J. Hartigan,et al.  A Bayesian Analysis for Change Point Problems , 1993 .

[17]  Takafumi Kanamori,et al.  Density Ratio Estimation in Machine Learning , 2012 .

[18]  P. Fearnhead,et al.  Optimal detection of changepoints with a linear computational cost , 2011, 1101.1438.

[19]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[20]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[21]  G ShinKang,et al.  Change-Point Monitoring for the Detection of DoS Attacks , 2004 .

[22]  Piotr Fryzlewicz,et al.  Wild binary segmentation for multiple change-point detection , 2014, 1411.0858.

[23]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[24]  L. Bottou Stochastic Gradient Learning in Neural Networks , 1991 .

[25]  Daniela Jarušková,et al.  Some Problems with Application of Change-Point Detection Methods to Environmental Data , 1997 .

[26]  B. Brodsky,et al.  Nonparametric Methods in Change Point Problems , 1993 .

[27]  Masashi Sugiyama,et al.  Sequential change‐point detection based on direct density‐ratio estimation , 2012, Stat. Anal. Data Min..

[28]  Fredrik Gustafsson,et al.  Adaptive filtering and change detection , 2000 .

[29]  Robert Lund,et al.  A Review and Comparison of Changepoint Detection Techniques for Climate Data , 2007 .

[30]  D. A. Hsu,et al.  A Bayesian Robust Detection of Shift in the Risk Structure of Stock Market Returns , 1982 .

[31]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[32]  Kenji Yamanishi,et al.  A unifying framework for detecting outliers and change points from non-stationary time series data , 2002, KDD.

[33]  B. Ray,et al.  Bayesian methods for change‐point detection in long‐range dependent processes , 2002 .

[34]  Tsuyoshi Idé,et al.  Change-Point Detection using Krylov Subspace Learning , 2007, SDM.

[35]  José Vinícius de Miranda Cardoso,et al.  Anomalies Detection in Wireless Sensor Networks Using Bayesian Changepoints , 2016, 2016 IEEE 13th International Conference on Mobile Ad Hoc and Sensor Systems (MASS).

[36]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[37]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[38]  Takafumi Kanamori,et al.  Statistical outlier detection using direct density ratio estimation , 2011, Knowledge and Information Systems.

[39]  Steffen Bickel,et al.  Discriminative learning for differing training and test distributions , 2007, ICML '07.

[40]  Yanchun Liang,et al.  Non-Parametric Change-Point Method for Differential Gene Expression Detection , 2011, PloS one.