Copula-Type Estimators for Flexible Multivariate Density Modeling Using Mixtures

Copulas are popular as models for multivariate dependence because they allow the marginal densities and the joint dependence to be modeled separately. However, they usually require that the transformation from uniform marginals to the marginals of the joint dependence structure is known. This can only be done for a restricted set of copulas, for example, a normal copula. Our article introduces copula-type estimators for flexible multivariate density estimation which also allow the marginal densities to be modeled separately from the joint dependence, as in copula modeling, but overcomes the lack of flexibility of most popular copula estimators. An iterative scheme is proposed for estimating copula-type estimators and its usefulness is demonstrated through simulation and real examples. The joint dependence is modeled by mixture of normals and mixture of normal factor analyzer models, and mixture of t and mixture of t-factor analyzer models. We develop efficient variational Bayes algorithms for fitting these in which model selection is performed automatically. Based on these mixture models, we construct four classes of copula-type densities which are far more flexible than current popular copula densities, and outperform them in a simulated dataset and several real datasets. Supplementary material for this article is available online.

[1]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  D. F. Andrews,et al.  Scale Mixtures of Normal Distributions , 1974 .

[3]  George E. P. Box,et al.  Sampling and Bayes' inference in scientific modelling and robustness , 1980 .

[4]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[5]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[6]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[7]  Satishs Iyengar,et al.  Multivariate Models and Dependence Concepts , 1998 .

[8]  Joachim Weickert,et al.  Scale-Space Theories in Computer Vision , 1999, Lecture Notes in Computer Science.

[9]  Zoubin Ghahramani,et al.  Variational Inference for Bayesian Mixtures of Factor Analysers , 1999, NIPS.

[10]  Bill Ravens,et al.  An Introduction to Copulas , 2000, Technometrics.

[11]  Geoffrey J. McLachlan,et al.  Robust mixture modelling using the t distribution , 2000, Stat. Comput..

[12]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[13]  Adrian Corduneanu,et al.  Variational Bayesian Model Selection for Mixture Distributions , 2001 .

[14]  Jack C. Lee,et al.  Bayesian analysis of mixture modelling using the multivariate t distribution , 2004, Stat. Comput..

[15]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[16]  Geoffrey J. McLachlan,et al.  Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution , 2007, Comput. Stat. Data Anal..

[17]  D. M. Titterington,et al.  Variational approximations in Bayesian model selection for finite mixture distributions , 2007, Comput. Stat. Data Anal..

[18]  Ajay Jasra,et al.  Population-Based Reversible Jump Markov Chain Monte Carlo , 2007, 0711.0186.

[19]  R. Kohn,et al.  Flexible Multivariate Density Estimation With Marginal Adaptation , 2009, 0901.0225.

[20]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[21]  M. Wand,et al.  Explaining Variational Approximations , 2010 .

[22]  Pavel Krupskii,et al.  Factor copula models for multivariate data , 2013, J. Multivar. Anal..