Hierarchical Similarity Transformations Between Gaussian Mixtures

In this paper, we propose a method to estimate the density of a data space represented by a geometric transformation of an initial Gaussian mixture model. The geometric transformation is hierarchical, and it is decomposed into two steps. At first, the initial model is assumed to undergo a global similarity transformation modeled by translation, rotation, and scaling of the model components. Then, to increase the degrees of freedom of the model and allow it to capture fine data structures, each individual mixture component may be transformed by another, local similarity transformation, whose parameters are distinct for each component of the mixture. In addition, to constrain the order of magnitude of the local transformation (LT) with respect to the global transformation (GT), zero-mean Gaussian priors are imposed onto the local parameters. The estimation of both GT and LT parameters is obtained through the expectation maximization framework. Experiments on artificial data are conducted to evaluate the proposed model, with varying data dimensionality, number of model components, and transformation parameters. In addition, the method is evaluated using real data from a speech recognition task. The obtained results show a high model accuracy and demonstrate the potential application of the proposed method to similar classification problems.

[1]  Aristidis Likas,et al.  Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting , 2007, IEEE Transactions on Neural Networks.

[2]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[3]  Nikolas P. Galatsanos,et al.  A Bayesian Framework for Image Segmentation With Spatially Varying Mixtures , 2010, IEEE Transactions on Image Processing.

[4]  M. Aitkin Likelihood and Bayesian analysis of mixtures , 2001 .

[5]  Vassilios Digalakis,et al.  Speaker adaptation using combined transformation and Bayesian methods , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[6]  Jie Yang,et al.  Variational Bayesian method for speech enhancement , 2007, Neurocomputing.

[7]  Juan Manuel Sáez,et al.  Color Image Segmentation Through Unsupervised Gaussian Mixture Models , 2006, IBERAMIA-SBIA.

[8]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[9]  Vassilios Digalakis,et al.  Speaker adaptation using combined transformation and Bayesian methods , 1996, IEEE Trans. Speech Audio Process..

[10]  Nikolas P. Galatsanos,et al.  Spatially Varying Mixtures Incorporating Line Processes for Image Segmentation , 2009, Journal of Mathematical Imaging and Vision.

[11]  J. Berger,et al.  Estimation of a Covariance Matrix Using the Reference Prior , 1994 .

[12]  Jinwen Ma,et al.  The BYY annealing learning algorithm for Gaussian mixture with automated model selection , 2007, Pattern Recognit..

[13]  Mark J. F. Gales,et al.  Maximum margin training of generative kernels , 2004 .

[14]  Jinwen Ma,et al.  Asymptotic Convergence Rate of the EM Algorithm for Gaussian Mixtures , 2000, Neural Computation.

[15]  Louis ten Bosch,et al.  Speaker normalization for automatic speech recognition — An on-line approach , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[16]  Adrian E. Raftery,et al.  Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering , 2007, J. Classif..

[17]  Michael I. Jordan,et al.  On Convergence Properties of the EM Algorithm for Gaussian Mixtures , 1996, Neural Computation.

[18]  P. Deb Finite Mixture Models , 2008 .

[19]  Nikolas P. Galatsanos,et al.  Majorization-minimization mixture model determination in image segmentation , 2011, CVPR 2011.

[20]  Baba C. Vemuri,et al.  A robust algorithm for point set registration using mixture of Gaussians , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[21]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[22]  Douglas A. Reynolds,et al.  Speaker Verification Using Adapted Gaussian Mixture Models , 2000, Digit. Signal Process..

[23]  Murray Aitkin,et al.  A general maximum likelihood analysis of overdispersion in generalized linear models , 1996, Stat. Comput..

[24]  Nikos A. Vlassis,et al.  A Greedy EM Algorithm for Gaussian Mixture Learning , 2002, Neural Processing Letters.

[25]  Chao Feng,et al.  Dynamical Gaussian mixture model for tracking elliptical living objects , 2006, Pattern Recognit. Lett..

[26]  Jinwen Ma,et al.  On the correct convergence of the EM algorithm for Gaussian mixtures , 2005, Pattern Recognit..

[27]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[28]  Nikolas P. Galatsanos,et al.  A spatially constrained mixture model for image segmentation , 2005, IEEE Transactions on Neural Networks.

[29]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[30]  Aristidis Likas,et al.  The mixtures of Student's t-distributions as a robust framework for rigid registration , 2009, Image Vis. Comput..

[31]  Sami S. Brandt,et al.  Maximum Likelihood Robust Regression by Mixture Models , 2006, Journal of Mathematical Imaging and Vision.

[32]  L. A. Goodman Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[33]  Shinji Watanabe,et al.  Automatic determination of acoustic model topology using variational Bayesian estimation and clustering for large vocabulary continuous speech recognition , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[34]  Vassilios Digalakis,et al.  Speaker adaptation using constrained estimation of Gaussian mixtures , 1995, IEEE Trans. Speech Audio Process..

[35]  David J. Miller,et al.  Combined Learning and Use for a Mixture Model Equivalent to the RBF Classifier , 1998, Neural Computation.

[36]  Edwin R. Hancock,et al.  Cartographic matching with millimetre radar images , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.