Automatic Change-Point Detection in Time Series via Deep Learning

Detecting change-points in data is challenging because of the range of possible types of change and types of behaviour of data when there is no change. Statistically efficient methods for detecting a change will depend on both of these features, and it can be difficult for a practitioner to develop an appropriate detection method for their application of interest. We show how to automatically generate new offline detection methods based on training a neural network. Our approach is motivated by many existing tests for the presence of a change-point being representable by a simple neural network, and thus a neural network trained with sufficient data should have performance at least as good as these methods. We present theory that quantifies the error rate for such an approach, and how it depends on the amount of training data. Empirical results show that, even with limited training data, its performance is competitive with the standard CUSUM-based classifier for detecting a change in mean when the noise is independent and Gaussian, and can substantially outperform it in the presence of auto-correlated or heavy-tailed noise. Our method also shows strong results in detecting and localising changes in activity based on accelerometer data.

[1]  Xiuyuan Cheng,et al.  Training Neural Networks for Sequential Change-point Detection , 2023, ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Tengyao Wang,et al.  Robust mean change point testing in high-dimensional data with heavy tails , 2023, 2305.18987.

[3]  Rajesh Wadhvani,et al.  Real-time Change-Point Detection: A deep neural network-based adaptive approach for detecting changes in multivariate time series data , 2022, Expert Syst. Appl..

[4]  Peter Buhlmann,et al.  Random Forests for Change Point Detection , 2022, J. Mach. Learn. Res..

[5]  P. Fryzlewicz.  Robust Narrowest Significance Pursuit: inference for multiple change-points in the median , 2021, 2109.02487.

[6]  Johannes Schmidt-Hieber,et al.  Convergence rates of deep ReLU networks for multiclass classification , 2021, Electronic Journal of Statistics.

[7]  P. Fryzlewicz,et al.  Cross-covariance isolate detect: a new change-point method for estimating dynamic functional connectivity , 2020, bioRxiv.

[8]  Toby Dylan Hocking,et al.  Increased peak detection accuracy in over-dispersed ChIP-seq data with supervised segmentation models , 2020, BMC Bioinform..

[9]  M. Lerasle,et al.  Optimal change-point detection and localization , 2020, The Annals of Statistics.

[10]  P. Fryzlewicz.  Narrowest Significance Pursuit: inference for multiple change-points in linear models , 2020, Journal of the American Statistical Association.

[11]  Alexander Bertrand,et al.  Change Point Detection in Time Series Data Using Autoencoders With a Time-Invariant Representation , 2020, IEEE Transactions on Signal Processing.

[12]  Paul Fearnhead,et al.  Relating and comparing methods for detecting changes in mean , 2020, Stat.

[13]  B. Póczos,et al.  Kernel Change-point Detection with Auxiliary Deep Generative Models , 2018, ICLR.

[14]  Farzaneh Ahmadzadeh,et al.  Change point detection with multivariate control charts by artificial neural network , 2018 .

[15]  Zuofeng Shang,et al.  Variance Change Point Detection Under a Smoothly-Changing Mean Trend with Application to Liver Procurement , 2018, Journal of the American Statistical Association.

[16]  Richard K. G. Do,et al.  Convolutional neural networks: an overview and application in radiology , 2018, Insights into Imaging.

[17]  Arun K. Kuchibhotla,et al.  Moving Beyond Sub-Gaussianity in High-Dimensional Statistics: Applications in Covariance Estimation and Linear Regression , 2018, 1804.02605.

[18]  Claudia Kirch,et al.  A MOSUM procedure for the estimation of multiple random change points , 2018 .

[19]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[20]  Johannes Schmidt-Hieber,et al.  Nonparametric regression using deep neural networks with ReLU activation function , 2017, The Annals of Statistics.

[21]  Diane J. Cook,et al.  Using change point detection to automate daily activity segmentation , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[22]  Peter L. Bartlett,et al.  Nearly-tight VC-dimension and Pseudodimension Bounds for Piecewise Linear Neural Networks , 2017, J. Mach. Learn. Res..

[23]  Adam N. Letchford,et al.  Detecting Changes in Slope With an L0 Penalty , 2017, Journal of Computational and Graphical Statistics.

[24]  P. Fearnhead,et al.  Computationally Efficient Changepoint Detection for a Range of Penalties , 2017 .

[25]  P. Fryzlewicz,et al.  Narrowest‐over‐threshold detection of multiple change points and change‐point‐like features , 2016, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[26]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Toby Hocking,et al.  PeakSeg: constrained optimal segmentation and supervised penalty learning for peak detection in count data , 2015, ICML.

[29]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Piotr Fryzlewicz,et al.  Wild binary segmentation for multiple change-point detection , 2014, 1411.0858.

[33]  Jukka Corander,et al.  Likelihood-free inference via classification , 2014, Statistics and Computing.

[34]  S. Fotopoulos,et al.  Inference for single and multiple change‐points in time series , 2013 .

[35]  H. Dehling,et al.  CHANGE-POINT DETECTION UNDER DEPENDENCE BASED ON TWO-SAMPLE U-STATISTICS , 2013, 1304.2479.

[36]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[37]  P. Fearnhead,et al.  Optimal detection of changepoints with a linear computational cost , 2011, 1101.1438.

[38]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[39]  Robert Lund,et al.  A Review and Comparison of Changepoint Detection Techniques for Climate Data , 2007 .

[40]  Tae Yoon Kim,et al.  Variance change point detection via artificial neural networks for data separation , 2005, Neurocomputing.

[41]  Franck Picard,et al.  A statistical approach for array CGH data analysis , 2005, BMC Bioinformatics.

[42]  A. Ng.  Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[43]  L. Breiman.  Random Forests , 2001, Encyclopedia of Machine Learning and Data Mining.

[44]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[45]  D. Siegmund,et al.  Tests for a change-point , 1987 .

[46]  N. Vayatis,et al.  Selective review of offline change point detection methods , 2019 .

[47]  Tengyao Wang,et al.  High-dimensional changepoint estimation via sparse projection , 2016 .

[48]  Zheng Tian,et al.  Variance change-point detection in panel data models , 2015 .

[49]  Alon Gonen.  Understanding Machine Learning From Theory to Algorithms 1st Edition Shwartz Solutions Manual , 2015 .

[50]  M. Gutmann,et al.  Approximate Bayesian Computation , 2012 .