Variational Bayesian inference for a Dirichlet process mixture of beta distributions and application

Abstract Finite beta mixture model (BMM) has been shown to be very flexible and powerful for bounded support data modeling. However, BMM cannot automatically select the proper number of the mixture components based on the observed data, which is important and has a deterministic effect on the modeling accuracy. In this paper, we aim at tackling this problem by infinite Beta mixture model (InBMM). It is based on the Dirichlet process (DP) mixture with the assumption that the number of the mixture components is infinite in advance and can be automatically determined according to the observed data. Further, a variational InBMM using single lower-bound approximation (VBInBMM) is proposed which applies the stick-breaking representation of the DP and is learned by an extended variational inference framework. Numerical experiments on both synthetic and real data, generated from two challenging application namely image categorization and object detection, demonstrate good performance obtained by the proposed method.

[1]  Tsung I. Lin,et al.  Maximum likelihood estimation for multivariate skew normal mixture models , 2009, J. Multivar. Anal..

[2]  Urbano Nunes,et al.  Trainable classifier-fusion schemes: An application to pedestrian detection , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[3]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[4]  Jinwen Ma,et al.  BYY harmony learning of log-normal mixtures with automated model selection , 2015, Neurocomputing.

[5]  Jun Guo,et al.  Dirichlet mixture modeling to estimate an empirical lower bound for LSF quantization , 2014, Signal Process..

[6]  Zhen Yang,et al.  The infinite Student's t-factor mixture analyzer for robust clustering and classification , 2012, Pattern Recognit..

[7]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Jalil Taghia,et al.  Bayesian Estimation of the von-Mises Fisher Mixture Model with Variational Inference , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nizar Bouguila,et al.  Variational Learning of Dirichlet Process Mixtures of Generalized Dirichlet Distributions and Its Applications , 2012, ADMA.

[10]  Germano C. Vasconcelos,et al.  Object recognition under severe occlusions with a hidden Markov model approach , 2017, Pattern Recognit. Lett..

[11]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Nizar Bouguila,et al.  Hybrid Generative/Discriminative Approaches for Proportional Data Modeling and Classification , 2012, IEEE Transactions on Knowledge and Data Engineering.

[13]  Jun Guo,et al.  Feature selection for neutral vector in EEG signal classification , 2016, Neurocomputing.

[14]  Nizar Bouguila,et al.  Infinite Dirichlet mixture models learning via expectation propagation , 2013, Adv. Data Anal. Classif..

[15]  Jonathan M. Garibaldi,et al.  Robust mixture clustering using Pearson type VII distribution , 2010, Pattern Recognit. Lett..

[16]  Babak Nadjar Araabi,et al.  Improved Bayesian information criterion for mixture model selection , 2016, Pattern Recognit. Lett..

[17]  Hiroshi Nakagawa,et al.  Quantum annealing for Dirichlet process mixture models with applications to network clustering , 2013, Neurocomputing.

[18]  Aurélien Garivier,et al.  A minimum description length approach to hidden Markov models with Poisson and Gaussian emissions. Application to order identification , 2009 .

[19]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[20]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[21]  Nizar Bouguila,et al.  Online Learning of a Dirichlet Process Mixture of Beta-Liouville Distributions Via Variational Inference , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Arne Leijon,et al.  Bayesian Estimation of Beta Mixture Models with Variational Inference , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nizar Bouguila,et al.  On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling , 2009, Pattern Analysis and Applications.

[24]  Paul A. Viola,et al.  Structure Driven Image Database Retrieval , 1997, NIPS.

[25]  Markus Flierl,et al.  Bayesian estimation of Dirichlet mixture model with variational inference , 2014, Pattern Recognit..

[26]  Adrian E. Raftery,et al.  Approximate Bayes Factors for Image Segmentation: The Pseudolikelihood Information Criterion (PLIC) , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[28]  Jean-Yves Tourneret,et al.  Classification of chirp signals using hierarchical Bayesian learning and MCMC methods , 2002, IEEE Trans. Signal Process..

[29]  Chunguang Li,et al.  The infinite Student's t-mixture for robust modeling , 2012, Signal Process..

[30]  Perry R. Cook,et al.  Bayesian Nonparametric Matrix Factorization for Recorded Music , 2010, ICML.

[31]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[32]  Jun Guo,et al.  Line spectral frequencies modeling by a mixture of von Mises-Fisher distributions , 2015, Signal Process..

[33]  Honggang Zhang,et al.  Variational Bayesian Matrix Factorization for Bounded Support Data , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Zhen Yang,et al.  Decorrelation of Neutral Vector Variables: Theory and Applications , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Chunguang Li,et al.  The Student's $t$-Hidden Markov Model With Truncated Stick-Breaking Priors , 2011, IEEE Signal Processing Letters.

[36]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[37]  Shimon Ullman,et al.  Learning to Segment , 2004, ECCV.

[38]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[39]  Jon D. McAuliffe,et al.  Variational Inference for Large-Scale Models of Discrete Choice , 2007, 0712.2526.

[40]  Douglas A. Reynolds,et al.  Robust text-independent speaker identification using Gaussian mixture speaker models , 1995, IEEE Trans. Speech Audio Process..

[41]  Lihong Zhu,et al.  Color texture image retrieval based on Gaussian copula models of Gabor wavelets , 2017, Pattern Recognit..

[42]  Nizar Bouguila,et al.  Variational learning of finite Dirichlet mixture models using component splitting , 2014, Neurocomputing.

[43]  R. K. Agrawal,et al.  A convex hull approach in conjunction with Gaussian mixture model for salient object detection , 2016, Digit. Signal Process..

[44]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[45]  Arne Leijon,et al.  Vector quantization of LSF parameters with a mixture of dirichlet distributions , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[46]  Haruhiko Ogasawara,et al.  Bias correction of the Akaike information criterion in factor analysis , 2016, J. Multivar. Anal..

[47]  Sami Bourouis,et al.  Bayesian learning of finite generalized inverted Dirichlet mixtures: Application to object classification and forgery detection , 2014, Expert Syst. Appl..

[48]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[49]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[51]  Q. M. Jonathan Wu,et al.  Robust Student's-t Mixture Model With Spatial Constraints and Its Application in Medical Image Segmentation , 2012, IEEE Transactions on Medical Imaging.

[52]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[53]  Florence Forbes,et al.  A new family of multivariate heavy-tailed distributions with variable marginal amounts of tailweight: application to robust clustering , 2013, Statistics and Computing.