Bayesian Multisensory Perception and Tracking

Humans and machines equipped with multiple sensor modalities need to combine information from various senses to obtain an accurate, unified perception of the world. Previous research has addressed statistically optimal fusion of multisensory observations of a given object[2,3]. However, in most real world situations any given pair of observations are unlikely to have originated from the same latent source. A more general problem in multi-sensor perception is therefore to infer the association between observations and any latent states of interest as well as any fusion (integration) or fission (segregation) that may be necessary. In some domains, these causal, association variables may also have critical independent meaning. Example To fully understand a meeting, two sets of latent object states (who was there & what was said) and the data association (who said what) must be inferred.