The /spl alpha/-EM learning and its cookbook: from mixture-of-expert neural networks to movie random field

The /spl alpha/-EM algorithm is a proper extension of the traditional log-EM algorithm. This new algorithm is based on the /spl alpha/-logarithm, while the traditional one uses the logarithm. The case of /spl alpha/=-1 corresponds to the log-EM algorithm. Since the speed of the /spl alpha/-EM algorithm was reported for learning problems, this paper shows that closed-form E-steps can be obtained for a wide class of problems. There is a set of common techniques. That is, a cookbooks for the /spl alpha/-EM algorithm is presented. The recipes include unsupervised neural networks, supervised neural networks for various gating, hidden Markov models and Markov random fields for moving object segmentation. Reasoning for the speedup is also given.

[1]  Nuno Vasconcelos,et al.  Empirical Bayesian EM-based motion segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Y. Matsuyama,et al.  The weighted EM algorithm and block monitoring , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[3]  Yasuo Matsuyama,et al.  The Alpha-EM Algorithm: A Block Connectable Generalized Learning Tool for Neural Networks , 1997, IWANN.

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[6]  T. Ikeda,et al.  Prior Probability Weights, Neural Network Learning , 1997, International Conference on Neural Information Processing.

[7]  Michael I. Jordan,et al.  On Convergence Properties of the EM Algorithm for Gaussian Mixtures , 1996, Neural Computation.

[8]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[9]  Geoffrey E. Hinton,et al.  An Alternative Model for Mixtures of Experts , 1994, NIPS.

[10]  Yasuo Matsuyama.  Learning Speed Evaluation of the α-EM Algorithm , 1999 .

[11]  Yasuo Matsuyama,et al.  Non-logarithmic information measures, /spl alpha/-weighted EM algorithms and speedup of learning , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[12]  Yasuo Matsuyama,et al.  WEM Algorithms and Probabilistic Learning , 1998 .

[13]  松山 泰男.  The Weighted EM Learning and Monitoring Structure , 1997 .

[14]  Naoki Takeda,et al.  A Hierarchy from alfa-EM Algorithm to Vector Quantization and Self Organization , 1998, ICONIP.

[15]  Michael I. Jordan,et al.  Convergence results for the EM approach to mixtures of experts architectures , 1995, Neural Networks.