Incremental Learning of Multivariate Gaussian Mixture Models

This paper presents a new algorithm for unsupervised incremental learning based on a Bayesian framework. The algorithm, called IGMM (for Incremental Gaussian Mixture Model), creates and continually adjusts a Gaussian Mixture Model consistent to all sequentially presented data. IGMM is particularly useful for on-line incremental clustering of data streams, as encountered in the domain of mobile robotics and animats. It creates an incremental knowledge model of the domain consisting of primitive concepts involving all observed variables. We present some preliminary results obtained using synthetic data and also consider practical issues as convergence properties discuss future developments.

[1]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[2]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[3]  Matej Kristan.  Incremental learning with Gaussian mixture models , 2008 .

[4]  Vipin Kumar,et al.  Introduction to Data Mining, (First Edition) , 2005 .

[5]  Douglas H. Fisher,et al.  Knowledge Acquisition Via Incremental Conceptual Clustering , 1987, Machine Learning.

[6]  Eric Moulines,et al.  On‐line expectation–maximization algorithm for latent data models , 2007, ArXiv.

[7]  Daniel G. Keehn,et al.  A note on learning for Gaussian properties , 1965, IEEE Trans. Inf. Theory.

[8]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[9]  Roberto Cipolla,et al.  Incremental Learning of Temporally-Coherent Gaussian Mixture Models , 2005, BMVC.

[10]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[11]  Eric Moulines,et al.  Recursive Em Algorithm with Applications to Doa Estimation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[12]  Douglas Hayes Fisher,et al.  Knowledge acquisition via incremental conceptual clustering : a dussertation submitted in partial satisfaction of the requirements for the degree doctor of philosophy in information and computer science , 1987 .

[13]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[14]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[15]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[16]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[17]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[18]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[19]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[20]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .

[21]  Shaojun Wang,et al.  Almost sure convergence of Titterington's recursive estimator for mixture models , 2002, Proceedings IEEE International Symposium on Information Theory,.