Split-Merge Incremental LEarning (SMILE) of Mixture Models

In this article we present an incremental method for building a mixture model. Given the desired number of clusters K ≥ 2, we start with a two-component mixture and we optimize the likelihood by repeatedly applying a Split-Merge operation. When an optimum is obtained, we add a new component to the model by splitting in two, a properly chosen cluster. This goes on until the number of components reaches a preset limiting value. We have performed numerical experiments on several data-sets and report a performance comparison with other rival methods.

[1]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[2]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[3]  Andrew R. Barron,et al.  Mixture Density Estimation , 1999, NIPS.

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[6]  Alan C. Evans,et al.  BrainWeb: Online Interface to a 3D MRI Simulated Brain Database , 1997 .

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Nikos A. Vlassis,et al.  A Greedy EM Algorithm for Gaussian Mixture Learning , 2002, Neural Processing Letters.

[9]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[10]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[11]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[12]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[13]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[14]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .