Generative Probabilistic Modeling: Understanding Causal Sensorimotor Integration

In this chapter, we argue that many aspects of human perception are best explained by adopting a modeling approach in which experimental subjects are assumed to possess a full generative probabilistic model of the task they are faced with, and that they use this model to make inferences about their environment and act optimally given the information available to them. We apply this generative modeling framework in two diverse settings—concurrent sensory andmotor adaptation, andmultisensory oddity detection—and show, in both cases, that the data are best described by a full generative modeling approach. Bayesian ideal-observer modeling is an elegant and successful normative approach to understanding human perception. One particular domain in which it has seen much success recently is that of understanding multisensory integration in human perception (see Chapter 1). Existing applications of this modeling approach have frequently focused on a simple special case where the ideal observer’s estimate of an unknown quantity in the environment is a reliability-weighted mean of the individual observed cues. This is all that is needed to understand a wide variety of interesting perceptual phenomena. We argue, however, that the Bayesian-observer approach can be more powerfully and generally applied by clear generative modeling of the perceptual task for each experiment. In other words, this assumes that people have access to a full generativemodel of their observations and that they use this model to make optimal decisions in performing the task. This systematic approach effectively provides a “model for modeling” that has some key advantages: (1) It provides the modeler with a clear framework for modeling new tasks beyond simply applying common normative models— such as linear combination—which may not apply for a new scenario and may fail to explain important aspects of human behavior; (2) Human performance can be measured against these clear “optimal”models such thatwe candraw conclusions about optimality of human perception or reveal architectural limitations of the human perceptual system, which cause it to deviate from optimality. For a particular perceptual task, the optimal solution requires inference in the true generative model of the task. Here, optimal is defined in the sense that the posterior probability over relevant unknowns in the environment is calculated. Any actions or decisions to bemade can then be taken