Mixtures of regressions with changepoints

We introduce an extension to the mixture of linear regressions model where changepoints are present. Such a model provides greater flexibility over a standard changepoint regression model if the data are believed to not only have changepoints present, but are also believed to belong to two or more unobservable categories. This model can provide additional insight into data that are already modeled using mixtures of regressions, but where the presence of changepoints has not yet been investigated. After discussing the mixture of regressions with changepoints model, we then develop an Expectation/Conditional Maximization (ECM) algorithm for maximum likelihood estimation. Two simulation studies illustrate the performance of our ECM algorithm and we analyze a real dataset.

[1]  J. Franke,et al.  Mixtures of nonparametric autoregressions , 2011 .

[2]  D. Hunter,et al.  Semiparametric mixtures of regressions , 2012 .

[3]  D. Hunter,et al.  mixtools: An R Package for Analyzing Mixture Models , 2009 .

[4]  G. McLachlan.  On Bootstrapping the Likelihood Ratio Test Statistic for the Number of Components in a Normal Mixture , 1987 .

[5]  Geoffrey J. McLachlan,et al.  Using the EM algorithm to train neural networks: misconceptions and a new algorithm for multiclass classification , 2004, IEEE Transactions on Neural Networks.

[6]  Matthew G Betts,et al.  Thresholds in Songbird Occurrence in Relation to Landscape Structure , 2007, Conservation biology : the journal of the Society for Conservation Biology.

[7]  Clifford M. Hurvich,et al.  Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion , 1998 .

[8]  Marc Henry,et al.  Identifying Finite Mixtures in Econometric Models , 2010 .

[9]  P. Sprent.  Some Hypotheses Concerning Two Phase Regression Lines , 1961 .

[10]  K. Worsley.  Testing for a Two-Phase Multiple Regression , 1983 .

[11]  J. B. Ramsey,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions , 1978 .

[12]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[13]  D. Rubin,et al.  Estimation and Hypothesis Testing in Finite Mixture Models , 1985 .

[14]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[15]  Xiao-Li Meng,et al.  Maximum likelihood estimation via the ECM algorithm: A general framework , 1993 .

[16]  Edit Gombay.  Change detection in autoregressive time series , 2008 .

[17]  C. Matias,et al.  Identifiability of parameters in latent structure models with many observed variables , 2008, 0809.5032.

[18]  J. Zidek,et al.  ON SEGMENTED MULTIVARIATE REGRESSION , 1997 .

[19]  T. Turner,et al.  Estimating the propagation rate of a viral infection of potato plants via mixtures of regressions , 2000 .

[20]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[21]  W. A. Shewhart,et al.  Statistical method from the viewpoint of quality control , 1939 .

[22]  D. Hinkley.  Inference about the intersection in two-phase regression , 1969 .

[23]  K Ulm,et al.  A statistical method for assessing a threshold in epidemiological studies. , 1991, Statistics in medicine.

[24]  R. D. Veaux,et al.  Mixtures of linear regressions , 1989 .

[25]  C. Robert,et al.  Estimating Mixtures of Regressions , 2003 .

[26]  G. C. Tiao,et al.  Identifying Mixtures of Regression Equations by the SAR procedure , 2003 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  W. DeSarbo,et al.  A Bayesian methodology for simultaneously detecting and estimating regime change points and variable selection in multiple regression models for marketing research , 2007 .

[29]  N. Brinkman.  Ethanol Fuel-A Single-Cylinder Engine Study of Efficiency and Exhaust Emissions , 1981 .

[30]  Achim Zeileis,et al.  Strucchange: An R package for testing for structural change in linear regression models , 2002 .

[31]  W. DeSarbo,et al.  A maximum likelihood methodology for clusterwise linear regression , 1988 .

[32]  Donald W. K. Andrews,et al.  Optimal changepoint tests for normal linear regression , 1996 .

[33]  F. Leisch.  FlexMix: A general framework for finite mixture models and latent class regression in R , 2004 .

[34]  Tristan Mary-Huard,et al.  ChIPmix: mixture model of regressions for two-color ChIP-chip analysis , 2008, ECCB.

[35]  V. Muggeo.  Estimating regression models with unknown break‐points , 2003, Statistics in medicine.

[36]  G. McLachlan,et al.  The EM Algorithm and Extensions: Second Edition , 2008 .

[37]  David R. Hunter,et al.  mixtools: An R Package for Analyzing Mixture Models , 2009 .

[38]  Marc Henry,et al.  Partial Identification of Finite Mixtures in Econometric Models , 2013 .

[39]  J. Neter,et al.  Applied Linear Regression Models , 1983 .

[40]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[41]  Kert Viele,et al.  Modeling with Mixtures of Linear Regressions , 2002, Stat. Comput..

[42]  Muni S. Srivastava,et al.  Regression Analysis: Theory, Methods, and Applications , 1991 .

[43]  Siddhartha Chib,et al.  Bayesian model selection for join point regression with application to age‐adjusted cancer rates , 2005 .

[44]  N. Kiefer.  Discrete Parameter Variation: Efficient Estimation of a Switching Regression Model , 1978 .

[45]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[46]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[47]  Jianhua Zhao,et al.  Fast ML Estimation for the Mixture of Factor Analyzers via an ECM Algorithm , 2008, IEEE Transactions on Neural Networks.

[48]  Dawei Huang,et al.  Testing for a Change in the Parameter Values and Order of an Autoregressive Model , 1995 .

[49]  X. Shao,et al.  Testing for Change Points in Time Series , 2010 .

[50]  M. Muggeo,et al.  segmented: An R package to Fit Regression Models with Broken-Line Relationships , 2008 .

[51]  Christian Hennig,et al.  Identifiablity of Models for Clusterwise Linear Regression , 2000, J. Classif..

[52]  S. Julious.  Inference and estimation in a changepoint regression problem , 2001 .

[53]  L. Horváth,et al.  Limit Theorems in Change-Point Analysis , 1997 .

[54]  Xiao-Li Meng,et al.  On the rate of convergence of the ECM algorithm , 1994 .

[55]  D. S. Young,et al.  Mixtures of regressions with predictor-dependent mixing proportions , 2010, Comput. Stat. Data Anal..

[56]  Woochul Kim,et al.  Estimation of a regression function with a sharp change point using boundary wavelets , 2004 .

[57]  R. Quandt.  A New Approach to Estimating Switching Regressions , 1972 .