November 21 , 2000 GCNU TR 2000 – 008 Products of Hidden Markov Models

We present products of hidden Markov models (PoHMM’s), a way of combining HMM’s to form a distributed state time series model. Inference in a PoHMM is tractable and efficient. Learning of the parameters, although intractable, can be effectively done using the Product of Experts learning rule. The distributed state helps the model to explain data which has multiple causes, and the fact that each model need only explain part of the data means a PoHMM can capture longer range structure than an HMM is capable of. We show some results on modelling character strings, a simple language task and the symbolic family trees problem, which highlight these advantages. Products of Hidden Markov Models Andrew D. Brown andy@gatsby.ucl.ac.uk Geoffrey E. Hinton hinton@gatsby.ucl.ac.uk