Bayesian data fusion and credit assignment in vision and fMRI data analysis

One of the most important challenges in understanding expert perception is in determining what information in a complex scene is most valuable (reliable) for a particular task, and how experts learn to exploit it. For the task of parameter estimation given multiple independent sources of data, Bayesian data fusion provides a solution to this problem that involves promoting data to a common parameter space and combining cues weighted by their reliabilities. However, for classification tasks this approach needs to be modified to find the information that most reliably distinguishes between the categories. In this paper we discuss solutions to the problem of determining the task-dependent reliability of data sources both objectively for a Bayesian decision agent, and in terms of the reliability assigned by a human observer from the performance of the observer. Modeling observers as Bayesian decision agents, solutions can be construed as a process of assigning credit to data sources based on their contribution to task performance. Applications of this approach to human perceptual data and the analysis of fMRI data will be presented.

[1]  Paul R. Schrater,et al.  Mechanisms of visual motion detection , 2000, Nature Neuroscience.

[2]  D. Knill.  Ideal observer perturbation analysis reveals human strategies for inferring surface orientation from texture , 1998, Vision Research.

[3]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[4]  D C Knill,et al.  Perception of surface contours and surface shape: from computation to psychophysics. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[5]  R Marken,et al.  Time and frequency analyses of auditory signal detection. , 1975, The Journal of the Acoustical Society of America.

[6]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[7]  Mark J. F. Gales,et al.  Maximum likelihood multiple projection schemes for hidden Markov models , 1999 .

[8]  S. McKee,et al.  Temporal coherence theory for the detection and measurement of visual motion , 1995, Vision Research.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Philippe G Schyns,et al.  Understanding Dali's Slave Market with the Disappearing Bust of Voltaire: A Case Study in the Scale Information Driving Perception , 2002, Perception.

[11]  Andrew Blake,et al.  Shape from texture: Ideal observers and human psychophysics , 1993, Vision Research.

[12]  D. Knill,et al.  Apparent surface curvature affects lightness perception , 1991, Nature.

[13]  P. Bennett,et al.  Identification of band-pass filtered letters and faces by human and ideal observers , 1999, Vision Research.

[14]  N. Campbell.  CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATION , 1984 .

[15]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .

[16]  Paul R. Schrater,et al.  Pattern inference theory: A probabilistic approach to vision , 2002 .

[17]  A. Yuille,et al.  Bayesian decision theory and psychophysics , 1996 .

[18]  Robert Tibshirani,et al.  Bias, Variance and Prediction Error for Classification Rules , 1996 .

[19]  A. Ahumada,et al.  Stimulus Features in Signal Detection , 1971 .

[20]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[21]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[22]  V. Richards,et al.  Relative estimates of combination weights, decision criteria, and internal noise based on correlation coefficients. , 1994, The Journal of the Acoustical Society of America.

[23]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[24]  Paul Robert Schrater,et al.  Local motion detection: Comparison of human and model observers , 1998 .

[25]  Leslie G. Ungerleider,et al.  Distributed representation of objects in the human ventral visual pathway. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[26]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[27]  Andreas G. Andreou,et al.  On Generalizations of Linear Discriminant Analysis , 1996 .