Multiple Change-Points Estimation in Linear Regression Models via Sparse Group Lasso

We consider linear regression problems for which the underlying model undergoes multiple changes. Our goal is to estimate the number and locations of change-points that segment available data into different regions, and further produce sparse and interpretable models for each region. To address challenges of the existing approaches and to produce interpretable models, we propose a sparse group Lasso based approach for linear regression problems with change-points. Under certain mild assumptions and a properly chosen regularization term, we prove that the solution of the proposed approach is asymptotically consistent. In particular, we show that the estimation error of linear coefficients diminishes, and the locations of the estimated change-points are close to those of true change-points. We further propose a method to choose the regularization term so that the results mentioned above hold. In addition, we show that the complexity of the proposed algorithm is much smaller than those of existing approaches. Numerical examples are provided to validate the analytical results.

[1]  P. McCullagh.  Partition models , 2015 .

[2]  Urbashi Mitra,et al.  Fast anomaly detection in SmartGrids via sparse approximation theory , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[3]  Guillem Rigaill,et al.  Pruned dynamic programming for optimal multiple change-point detection , 2010 .

[4]  Marc Lavielle,et al.  Using penalized contrasts for the change-point problem , 2005, Signal Process..

[5]  Lin Xiao,et al.  On the complexity analysis of randomized block-coordinate descent methods , 2013, Mathematical Programming.

[6]  Georgios B. Giannakis,et al.  Group lassoing change-points in piecewise-constant AR processes , 2012, EURASIP J. Adv. Signal Process..

[7]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[8]  Noah Simon,et al.  A Sparse-Group Lasso , 2013 .

[9]  Frederico R. B. Cruz,et al.  Multiple change-point analysis for linear regression models , 2010 .

[10]  Martin J. Wainwright,et al.  A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers , 2009, NIPS.

[11]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[12]  Jieping Ye,et al.  An efficient ADMM algorithm for multidimensional anisotropic total variation regularization problems , 2013, KDD.

[13]  T. Chan,et al.  Exact Solutions to Total Variation Regularization Problems , 1996 .

[14]  P. Perron,et al.  Computation and Analysis of Multiple Structural-Change Models , 1998 .

[15]  Junhui Qian,et al.  SHRINKAGE ESTIMATION OF REGRESSION MODELS WITH MULTIPLE STRUCTURAL CHANGES , 2015, Econometric Theory.

[16]  M. Drton,et al.  Exact block-wise optimization in group lasso and sparse group lasso for linear regression , 2010, 1010.3320.

[17]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[18]  J. Bai,et al.  Estimation of a Change Point in Multiple Regression Models , 1997, Review of Economics and Statistics.

[19]  Franck Picard,et al.  A statistical approach for array CGH data analysis , 2005, BMC Bioinformatics.

[20]  Yonina C. Eldar,et al.  Simultaneously Structured Models With Application to Sparse and Low-Rank Matrices , 2012, IEEE Transactions on Information Theory.

[21]  R. Bellman,et al.  Curve Fitting by Segmented Straight Lines , 1969 .

[22]  P. Rathinam,et al.  Energy use efficiency in sugarcane cultivation with respect to Erode district , 2016, BIOINFORMATICS 2016.

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[25]  Rebecca Willett,et al.  Online Optimization in Dynamic Environments , 2013, ArXiv.

[26]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[27]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[28]  Robert Lund,et al.  Multiple Changepoint Detection via Genetic Algorithms , 2012 .

[29]  P. Perron,et al.  Estimating and testing linear models with multiple structural changes , 1995 .

[30]  James D. B. Nelson,et al.  High dimensional changepoint detection with a dynamic graphical lasso , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  J. Bai,et al.  Likelihood ratio tests for multiple structural changes , 1999 .

[32]  H. Akaike.  A new look at the statistical model identification , 1974 .

[33]  Kenneth R. Davidson,et al.  Real Analysis and Applications: Theory in Practice , 2009 .

[34]  E. S. Page.  On problems in which a change in a parameter occurs at an unknown point , 1957 .

[35]  J. Janowiak,et al.  The Version 2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979-Present) , 2003 .

[36]  J. Hartigan,et al.  A Bayesian Analysis for Change Point Problems , 1993 .

[37]  Rebecca Willett,et al.  Change-Point Detection for High-Dimensional Time Series With Missing Data , 2012, IEEE Journal of Selected Topics in Signal Processing.

[38]  Z. Harchaoui,et al.  Multiple Change-Point Estimation With a Total Variation Penalty , 2010 .

[39]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.

[40]  Aleksey S. Polunchenko,et al.  State-of-the-Art in Sequential Change-Point Detection , 2011, 1109.2938.

[41]  P. K. Bhattacharya,et al.  Some aspects of change-point analysis , 1994 .

[42]  Taposh Banerjee,et al.  Quickest Change Detection , 2012, ArXiv.

[43]  David O Siegmund,et al.  A Modified Bayes Information Criterion with Applications to the Analysis of Comparative Genomic Hybridization Data , 2007, Biometrics.