Bayesian mixture of spatial spline regressions

This work relates the framework of model-based clustering for spatial functional data where the data are surfaces. We first introduce a Bayesian spatial spline regression model with mixed-effects (BSSR) for modeling spatial function data. The BSSR model is based on Nodal basis functions for spatial regression and accommodates both common mean behavior for the data through a fixed-effects part, and variability inter-individuals thanks to a random-effects part. Then, in order to model populations of spatial functional data issued from heterogeneous groups, we integrate the BSSR model into a mixture framework. The resulting model is a Bayesian mixture of spatial spline regressions with mixed-effects (BMSSR) used for density estimation and model-based surface clustering. The models, through their Bayesian formulation, allow to integrate possible prior knowledge on the data structure and constitute a good alternative to recent mixture of spatial spline regressions model estimated in a maximum likelihood framework via the expectation-maximization (EM) algorithm. The Bayesian model inference is performed by Markov Chain Monte Carlo (MCMC) sampling. We derive two Gibbs sampler to infer the BSSR and the BMSSR models and apply them on simulated surfaces and a real problem of handwritten digit recognition using the MNIST data set. The obtained results highlight the potential benefit of the proposed Bayesian approaches for modeling surfaces possibly dispersed in particular in clusters.

[1]  Radford M. Neal.  Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[2]  C. Robert.  The Bayesian choice : a decision-theoretic motivation , 1996 .

[3]  Aurore Delaigle,et al.  Componentwise classification and clustering of functional data , 2012 .

[4]  Kui Wang,et al.  A Mixture model with random-effects components for clustering correlated gene-expression profiles , 2006, Bioinform..

[5]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[6]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[7]  Faicel Chamroukhi,et al.  Hidden process regression for curve modeling, classification and tracking , 2010 .

[8]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[9]  George Casella,et al.  A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data , 2008, 0808.2902.

[10]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[11]  Julien Jacques,et al.  Model-based clustering for multivariate functional data , 2013, Comput. Stat. Data Anal..

[12]  Padhraic Smyth,et al.  Probabilistic curve-aligned clustering and prediction with regression mixture models , 2004 .

[13]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[14]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[15]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[16]  Geoffrey J. McLachlan,et al.  Mixture models for clustering multilevel growth trajectories , 2014, Comput. Stat. Data Anal..

[17]  Wayne S. DeSarbo,et al.  Bayesian inference for finite mixtures of generalized linear models with random effects , 2000 .

[18]  R. Fildes.  Journal of the American Statistical Association : William S. Cleveland, Marylyn E. McGill and Robert McGill, The shape parameter for a two variable graph 83 (1988) 289-300 , 1989 .

[19]  James O. Ramsay,et al.  Applied Functional Data Analysis: Methods and Case Studies , 2002 .

[20]  Adrian E. Raftery,et al.  Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering , 2007, J. Classif..

[21]  J. Ramsay,et al.  The historical functional linear model , 2003 .

[22]  J. O. Ramsay,et al.  Functional Data Analysis (Springer Series in Statistics) , 1997 .

[23]  Geoffrey J. McLachlan,et al.  Mixtures of spatial spline regressions for clustering and classification , 2016, Comput. Stat. Data Anal..

[24]  D. Hedeker,et al.  A RANDOM-EFFECTS MIXTURE MODEL FOR CLASSIFYING TREATMENT RESPONSE IN LONGITUDINAL CLINICAL TRIALS , 2001, Journal of biopharmaceutical statistics.

[25]  Xueli Liu,et al.  Simultaneous curve registration and clustering for functional data , 2009, Comput. Stat. Data Anal..

[26]  Hervé Glotin,et al.  Functional Mixture Discriminant Analysis with hidden process regression for curve classification , 2012, ESANN.

[27]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[28]  Allou Samé,et al.  A hidden process regression model for functional data description. Application to curve discrimination , 2010, Neurocomputing.

[29]  Frédéric Ferraty,et al.  Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics) , 2006 .

[30]  Allou Samé,et al.  Time series modeling by a regression approach based on a latent process , 2009, Neural Networks.

[31]  Catherine A. Sugar,et al.  Clustering for Sparsely Sampled Functional Data , 2003 .

[32]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[33]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[34]  Padhraic Smyth,et al.  Joint Probabilistic Curve Clustering and Alignment , 2004, NIPS.

[35]  P. Saama.  MAXIMUM LIKELIHOOD AND BAYESIAN METHODS FOR MIXTURES OF NORMAL DISTRIBUTIONS , 1997 .

[36]  G. Celeux,et al.  Mixture of linear mixed models for clustering gene expression profiles from repeated microarray experiments , 2005 .

[37]  Jean-Michel Marin,et al.  Bayesian Modelling and Inference on Mixtures of Distributions , 2005 .

[38]  James O. Ramsay,et al.  Spatial spline regression models , 2013 .

[39]  Volker Tresp,et al.  Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates , 1998, IEEE Trans. Neural Networks.

[40]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[41]  Gareth M. James,et al.  Functional linear discriminant analysis for irregularly sampled curves , 2001 .

[42]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[43]  Allou Samé,et al.  Model-based clustering and segmentation of time series with changes in regime , 2011, Adv. Data Anal. Classif..

[44]  Emilie Devijver,et al.  Model-based clustering for high-dimension data. Application to functional data. , 2014 .

[45]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[46]  Z. Q. John Lu,et al.  Nonparametric Functional Data Analysis: Theory And Practice , 2007, Technometrics.

[47]  James O. Ramsay,et al.  Spatial Functional Data Analysis , 2011 .

[48]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[49]  Christian P. Robert,et al.  The Bayesian choice : from decision-theoretic foundations to computational implementation , 2007 .

[50]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[51]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[52]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[53]  Emilie Devijver,et al.  Model-based regression clustering for high-dimensional data: application to functional data , 2014, Adv. Data Anal. Classif..