The brain uses adaptive internal models of scene statistics for sensorimotor estimation and planning

Significance To test the hypothesis that the brain uses adaptive models of object statistics to interpret sensory information, we measured the statistical models used by subjects to estimate object speed when asked to hit a moving object. Subjects’ behavior showed perceptual biases to the mean speed within a stimulus set that accurately adapted to changes in the variance of a stimulus set. More significantly, the results show that the way that stimuli on one trial influence following trials adapts appropriately to changes in trial-to-trial correlations in a stimulus set, although subjects’ estimates of correlations retain erroneous positive biases. Because of uncertainty and noise, the brain should use accurate internal models of the statistics of objects in scenes to interpret sensory signals. Moreover, the brain should adapt its internal models to the statistics within local stimulus contexts. Consider the problem of hitting a baseball. The impoverished nature of the visual information available makes it imperative that batters use knowledge of the temporal statistics and history of previous pitches to accurately estimate pitch speed. Using a laboratory analog of hitting a baseball, we tested the hypothesis that the brain uses adaptive internal models of the statistics of object speeds to plan hand movements to intercept moving objects. We fit Bayesian observer models to subjects’ performance to estimate the statistical environments in which subjects’ performance would be ideal and compared the estimated statistics with the true statistics of stimuli in an experiment. A first experiment showed that subjects accurately estimated and used the variance of object speeds in a stimulus set to time hitting behavior but also showed serial biases that are suboptimal for stimuli that were uncorrelated over time. A second experiment showed that the strength of the serial biases depended on the temporal correlations within a stimulus set, even when the biases were estimated from uncorrelated stimulus pairs subsampled from the larger set. Taken together, the results show that subjects adapted their internal models of the variance and covariance of object speeds within a stimulus set to plan interceptive movements but retained a bias to positive correlations.