Products of Experts

Note thatMixture of Expert Modelsare usually associated with conditional models where the experts are of the formp(y|x) and the mixture coefficients (known as gating functions) may depend on x as well, α(x). Conditional PoEs may be defined as well. One can qualitatively understand the difference between mixtures and products by observing that a mixture distribution can have high probability for event x when only a single expert assigns high probability to that event. In contrast, a product can only have high probability for an event x when all experts assign high probability to that event. Hence, metaphorically speaking, a single expert in a mixture has the power to pass a bill while a single expert in a product has the power to veto it. Put another way, each component in a product represents a soft constraint , while each expert in a mixture represents a soft template or prototype. For an event to be likely under a product model, all constraints must