Deep Gaussian mixture models

Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, deep Gaussian mixture models (DGMM) are introduced and discussed. A DGMM is a network of multiple layers of latent variables, where, at each layer, the variables follow a mixture of Gaussian distributions. Thus, the deep mixture model consists of a set of nested mixtures of linear models, which globally provide a nonlinear model able to describe the data in a very flexible way. In order to avoid overparameterized solutions, dimension reduction by factor models can be applied at each layer of the architecture, thus resulting in deep mixtures of factor analyzers.

[1]  Cinzia Viroli,et al.  Dimensionally Reduced Model-Based Clustering Through Mixtures of Factor Mixture Analyzers , 2010, J. Classif..

[2]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[3]  Volodymyr Melnykov.  Merging Mixture Components for Clustering Through Pairwise Overlap , 2016 .

[4]  SchmidhuberJürgen.  Deep learning in neural networks , 2015 .

[5]  Hal Hodson.  Facebook as good as a human at recognising faces , 2014 .

[6]  Gilles Celeux,et al.  Combining Mixture Components for Clustering , 2010, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[7]  Jia Li.  Clustering Based on a Multilayer Mixture Model , 2005 .

[8]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[9]  Benjamin Schrauwen,et al.  Factoring Variations in Natural Images with Deep Gaussian Mixture Models , 2014, NIPS.

[10]  Adrian E. Raftery,et al.  mclust Version 4 for R : Normal Mixture Modeling for Model-Based Clustering , Classification , and Density Estimation , 2012 .

[11]  Christian Hennig,et al.  Methods for merging Gaussian mixture components , 2010, Adv. Data Anal. Classif..

[12]  Geoffrey J. McLachlan,et al.  Modelling high-dimensional data by mixtures of factor analyzers , 2003, Comput. Stat. Data Anal..

[13]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[14]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[15]  Kui Wang,et al.  Multivariate Skew t Mixture Models: Applications to Fluorescence-Activated Cell Sorting Data , 2009, 2009 Digital Image Computing: Techniques and Applications.

[16]  Geoffrey J. McLachlan,et al.  Mixtures of Factor Analyzers with Common Factor Loadings: Applications to the Clustering and Visualization of High-Dimensional Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  M. Forina,et al.  Multivariate data analysis as a discriminating method of the origin of wines , 2015 .

[18]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[19]  Luca Scrucca,et al.  mclust 5: Clustering, Classification and Density Estimation Using Gaussian Finite Mixture Models , 2016, R J..

[20]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  A. Montanari,et al.  Heteroscedastic factor mixture analysis , 2010 .

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Geoffrey E. Hinton,et al.  Deep Mixtures of Factor Analysers , 2012, ICML.