On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling

In this paper, we present a fully Bayesian approach for generalized Dirichlet mixtures estimation and selection. The estimation of the parameters is based on the Monte Carlo simulation technique of Gibbs sampling mixed with a Metropolis-Hastings step. Also, we obtain a posterior distribution which is conjugate to a generalized Dirichlet likelihood. For the selection of the number of clusters, we used the integrated likelihood. The performance of our Bayesian algorithm is tested and compared with the maximum likelihood approach by the classification of several synthetic and real data sets. The generalized Dirichlet mixture is also applied to the problems of IR eye modeling and introduced as a probabilistic kernel for Support Vector Machines.

[1]  Irfan Essa,et al.  Detecting and tracking eyes by using their physiological properties , 2000, CVPR 2000.

[2]  Alfredo Petrosino,et al.  Finding Hidden Events in Astrophysical Data using PCA and Mixture of Gaussians Clustering , 2002, Pattern Analysis & Applications.

[3]  Riad I. Hammoud,et al.  Boosting particle filter-based eye tracker performance through adapted likelihood function to reflexions and light changes , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[4]  Nizar Bouguila,et al.  Online clustering via finite mixtures of Dirichlet and minimum message length , 2006, Eng. Appl. Artif. Intell..

[5]  Margrit Betke,et al.  Communication via eye blinks - detection and duration analysis in real time , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[7]  Nizar Bouguila,et al.  Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application , 2004, IEEE Transactions on Image Processing.

[8]  Roger Mohr,et al.  Mixture densities for video objects recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[9]  Riad I. Hammoud,et al.  Driver State Monitor from DELPHI , 2005, CVPR.

[10]  Efthymios G. Tsionas.  Bayesian inference for multivariate gamma distributions , 2004, Stat. Comput..

[11]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[12]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[13]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[14]  Trygve Randen,et al.  Filtering for Texture Classification: A Comparative Study , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Kee Tung. Wong,et al.  Texture features for image classification and retrieval. , 2002 .

[16]  D. Ziou,et al.  A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[17]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[18]  Siwei Lyu,et al.  How realistic is photorealistic , 2005 .

[19]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[20]  Christian P. Robert,et al.  Reversible Jump MCMC Converging to Birth-and-Death MCMC and More General Continuous Time Samplers , 2001 .

[21]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[22]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nizar Bouguila,et al.  On Fitting Finite Dirichlet Mixture Using ECM and MML , 2005, ICAPR.

[24]  O. Cappé,et al.  Reversible jump, birth‐and‐death and more general continuous time Markov chain Monte Carlo samplers , 2003 .

[25]  S. Sahu,et al.  A fast distance‐based approach for determining the number of components in mixtures , 2003 .

[26]  Nizar Bouguila,et al.  High-Dimensional Unsupervised Selection and Estimation of a Finite Generalized Dirichlet Mixture Model Based on Minimum Message Length , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[28]  C. Frankel,et al.  Distinguishing photographs and graphics on the World Wide Web , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[29]  N. L. Johnson,et al.  Continuous Multivariate Distributions: Models and Applications , 2005 .

[30]  Nizar Bouguila,et al.  Unsupervised learning of a finite discrete mixture: Applications to texture modeling and image databases summarization , 2007, J. Vis. Commun. Image Represent..

[31]  Riad I. Hammoud,et al.  An improved likelihood model for eye tracking , 2007, Comput. Vis. Image Underst..

[32]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[33]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[34]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[35]  Irfan A. Essa,et al.  Detecting and tracking eyes by using their physiological properties, dynamics, and appearance , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[36]  Nizar Bouguila,et al.  Dirichlet-based probability model applied to human skin detection [image skin detection] , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[37]  Gernot D. Kleiter,et al.  Bayesian Diagnosis in Expert Systems , 1992, Artif. Intell..

[38]  Nizar Bouguila,et al.  A hybrid SEM algorithm for high-dimensional unsupervised learning using a finite generalized Dirichlet mixture , 2006, IEEE Transactions on Image Processing.

[39]  Radford M. Neal.  Bayesian Mixture Modeling , 1992 .

[40]  Nizar Bouguila,et al.  A Bayesian Non-Gaussian Mixture Analysis: Application to Eye Modeling , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Nizar Bouguila,et al.  Practical Bayesian estimation of a finite beta mixture through gibbs sampling and its applications , 2006, Stat. Comput..

[42]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[43]  A. Raftery,et al.  Estimating Bayes Factors via Posterior Simulation with the Laplace—Metropolis Estimator , 1997 .

[44]  Nizar Bouguila,et al.  Novel Mixtures Based on the Dirichlet Distribution: Application to Data and Image Classification , 2003, MLDM.

[45]  Nizar Bouguila,et al.  Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach , 2006, IEEE Transactions on Knowledge and Data Engineering.

[46]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[47]  Zhiwei Zhu,et al.  Robust real-time eye detection and tracking under variable lighting conditions and various face orientations , 2005, Comput. Vis. Image Underst..

[48]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[49]  Dia I. Abu-Al-Nadi,et al.  Automated personal identification system based on human iris analysis , 2007, Pattern Analysis and Applications.

[50]  Jean-Michel Marin,et al.  Bayesian modelling and inference difficulties on mixtures of distributions , 2004 .

[51]  Enrique F. Castillo,et al.  Learning and Updating of Uncertainty in Dirichlet Models , 2004, Machine Learning.

[52]  Stephen P. Brooks,et al.  On Bayesian analyses and finite mixtures for proportions , 2001, Stat. Comput..

[53]  Mads Nielsen,et al.  Eye typing using Markov and active appearance models , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[54]  Jack C. Lee,et al.  Bayesian analysis of mixture modelling using the multivariate t distribution , 2004, Stat. Comput..

[55]  Takeo Kanade,et al.  Dual-state parametric eye tracking , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[56]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[57]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Ashok N. Srivastava,et al.  Mixture Density Mercer Kernels: A Method to Learn Kernels Directly from Data , 2004, SDM.

[59]  R. Beckman,et al.  Maximum likelihood estimation for the beta distribution , 1978 .

[60]  L. Wasserman,et al.  A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion , 1995 .

[61]  Horst Bunke,et al.  Towards Detection of Glasses in Facial Images , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[62]  B. Everitt,et al.  Finite Mixture Distributions , 1981 .

[63]  Nizar Bouguila,et al.  MML-Based Approach for High-Dimensional Unsupervised Learning Using the Generalized Dirichlet Mixture , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[64]  Michael Unser,et al.  Sum and Difference Histograms for Texture Classification , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  C. Robert,et al.  MCMC Control Spreadsheets for Exponential Mixture Estimation , 1999 .

[66]  Adrian E. Raftery,et al.  Inference in model-based cluster analysis , 1997, Stat. Comput..

[67]  Riad I. Hammoud,et al.  A robust eye position tracker based on invariant local features, eye motion, and infrared-eye responses , 2005, SPIE Defense + Commercial Sensing.

[68]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[69]  Jing Huang,et al.  Spatial Color Indexing and Applications , 2004, International Journal of Computer Vision.