Dynamic Causal Modelling of Active Vision

In this paper, we draw from recent theoretical work on active perception, which suggests that the brain makes use of an internal (i.e., generative) model to make inferences about the causes of sensations. This view treats visual sensations as consequent on action (i.e., saccades) and implies that visual percepts must be actively constructed via a sequence of eye movements. Oculomotor control calls on a distributed set of brain sources that includes the dorsal and ventral frontoparietal (attention) networks. We argue that connections from the frontal eye fields to ventral parietal sources represent the mapping from “where”, fixation location to information derived from “what” representations in the ventral visual stream. During scene construction, this mapping must be learned, putatively through changes in the effective connectivity of these synapses. Here, we test the hypothesis that the coupling between the dorsal frontal cortex and the right temporoparietal cortex is modulated during saccadic interrogation of a simple visual scene. Using dynamic causal modeling for magnetoencephalography with (male and female) human participants, we assess the evidence for changes in effective connectivity by comparing models that allow for this modulation with models that do not. We find strong evidence for modulation of connections between the two attention networks; namely, a disinhibition of the ventral network by its dorsal counterpart. SIGNIFICANCE STATEMENT This work draws from recent theoretical accounts of active vision and provides empirical evidence for changes in synaptic efficacy consistent with these computational models. In brief, we used magnetoencephalography in combination with eye-tracking to assess the neural correlates of a form of short-term memory during a dot cancellation task. Using dynamic causal modeling to quantify changes in effective connectivity, we found evidence that the coupling between the dorsal and ventral attention networks changed during the saccadic interrogation of a simple visual scene. Intuitively, this is consistent with the idea that these neuronal connections may encode beliefs about “what I would see if I looked there”, and that this mapping is optimized as new data are obtained with each fixation.

[1]  F. Tomaiuolo,et al.  The anatomy of neglect without hemianopia: a key role for parietal–frontal disconnection? , 2003, Neuroreport.

[2]  A. Yuille,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Vision as Bayesian inference: analysis by synthesis? , 2022 .

[3]  Dimitri Ognibene,et al.  Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot , 2015, IEEE Transactions on Autonomous Mental Development.

[4]  W. Fries Inputs from motor and premotor cortex to the superior colliculus of the macaque monkey , 1985, Behavioural Brain Research.

[5]  Igor Schindler,et al.  An exploration of the role of the superior temporal gyrus in visual search and spatial perception using TMS , 2014 .

[6]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[7]  Agatha Lenartowicz,et al.  Updating of context in working memory: An event-related potential study , 2010, Cognitive, affective & behavioral neuroscience.

[8]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[9]  M. Catani,et al.  A lateralized brain network for visuospatial attention , 2011, Nature Neuroscience.

[10]  Karl J. Friston,et al.  Working memory, attention, and salience in active inference , 2017, Scientific Reports.

[11]  Karl J. Friston,et al.  Dynamic causal modeling of evoked responses in EEG and MEG , 2006, NeuroImage.

[12]  David Vernon,et al.  Cognitive Vision: the Case for Embodied Perception , 2005 .

[13]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[14]  Rodrigo Montefusco-Siegmund,et al.  Saccades during visual exploration align hippocampal 3–8 Hz rhythms in human and non-human primates , 2013, Front. Syst. Neurosci..

[15]  K. Akert,et al.  Efferent connections of cortical, area 8 (frontal eye field) in Macaca fascicularis. A reinvestigation using the autoradiographic technique , 1977, The Journal of comparative neurology.

[16]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[17]  S. Shipp Structure and function of the cerebral cortex , 2007, Current Biology.

[18]  Karl J. Friston,et al.  Neural masses and fields in dynamic causal modeling , 2013, Front. Comput. Neurosci..

[19]  Xiaogang Yan,et al.  Visual–Motor Transformations Within Frontal Eye Fields During Head-Unrestrained Gaze Shifts in the Monkey , 2014, Cerebral cortex.

[20]  Karl J. Friston,et al.  Scene Construction, Visual Foraging, and Active Inference , 2016, Front. Comput. Neurosci..

[21]  K. Lovblad,et al.  Neuroanatomy of hemispatial neglect and its functional components: a study using voxel-based lesion-symptom mapping. , 2010, Brain : a journal of neurology.

[22]  Martin Lotze,et al.  Effects of inhibitory theta burst TMS to different brain sites involved in visuospatial attention - a combined neuronavigated cTBS and behavioural study. , 2016, Restorative neurology and neuroscience.

[23]  J. Lynch,et al.  The parieto‐collicular pathway: anatomical location and contribution to saccade generation , 2003, The European journal of neuroscience.

[24]  H. Karnath,et al.  The anatomy of spatial neglect , 2012, Neuropsychologia.

[25]  Karl J. Friston,et al.  The Computational Anatomy of Visual Neglect , 2017, Cerebral cortex.

[26]  Leslie G. Ungerleider,et al.  ‘What’ and ‘where’ in the human brain , 1994, Current Opinion in Neurobiology.

[27]  Scott Cheng-Hsin Yang,et al.  Active sensing in the categorization of visual patterns , 2016, eLife.

[28]  Karl J. Friston,et al.  Effective Connectivity Reveals Right-Hemisphere Dominance in Audiospatial Perception: Implications for Models of Spatial Neglect , 2014, The Journal of Neuroscience.

[29]  P. Goldman-Rakic,et al.  Preface: Cerebral Cortex Has Come of Age , 1991 .

[30]  Manfred Fahle,et al.  Dissociation of egocentric and allocentric coding of space in visual search after right middle cerebral artery stroke , 2008, Neuropsychologia.

[31]  Karl J. Friston,et al.  Dynamic causal modelling of evoked responses in EEG/MEG with lead field parameterization , 2006, NeuroImage.

[32]  Karl J. Friston,et al.  Posterior probability maps and SPMs , 2003, NeuroImage.

[33]  C. Bruce,et al.  Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. , 1985, Journal of neurophysiology.

[34]  J. Marshall,et al.  Neglect of Awareness , 1998, Consciousness and Cognition.

[35]  C. Pierrot-Deseilligny,et al.  Cortical control of saccades , 1998, Experimental Brain Research.

[36]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[37]  P. Bartolomeo,et al.  Left unilateral neglect as a disconnection syndrome. , 2007, Cerebral cortex.

[38]  Sabine Kastner,et al.  Thalamic functions in distributed cognitive control , 2017, Nature Neuroscience.

[39]  Karl J. Friston,et al.  The active construction of the visual world , 2017, Neuropsychologia.

[40]  Karl J. Friston,et al.  Active Inference: A Process Theory , 2017, Neural Computation.

[41]  Joachim Denzler,et al.  Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  R. Guillery,et al.  Thalamic Relay Functions and Their Role in Corticocortical Communication Generalizations from the Visual System , 2002, Neuron.

[43]  Karl J. Friston,et al.  Variational free energy and the Laplace approximation , 2007, NeuroImage.

[44]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[45]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[46]  Karl J. Friston,et al.  Computational Neuropsychology and Bayesian Inference , 2018, Front. Hum. Neurosci..

[47]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[48]  J. Bruineberg Active Inference and the Primacy of the ‘I Can’ , 2017 .

[49]  Karl J. Friston,et al.  Multiple sparse priors for the M/EEG inverse problem , 2008, NeuroImage.

[50]  C. Kennard,et al.  Impaired spatial working memory across saccades contributes to abnormal search in parietal neglect. , 2001, Brain : a journal of neurology.

[51]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[52]  Karl J. Friston,et al.  Deconstructing the Architecture of Dorsal and Ventral Attention Systems with Dynamic Causal Modeling , 2012, The Journal of Neuroscience.

[53]  M McCloskey,et al.  Attention-referenced visual representations: evidence from impaired visual localization. , 2000, Journal of experimental psychology. Human perception and performance.

[54]  R. Stout,et al.  ALBERT'S TEST: A NEGLECTED TEST OF PERCEPTUAL NEGLECT , 1986, The Lancet.

[55]  A. Yamadori,et al.  Dissociation of body-centered and stimulus-centered representations in unilateral neglect , 2001, Neurology.

[56]  H. Barlow Inductive Inference, Coding, Perception, and Language , 1974, Perception.

[57]  M. Albert A simple test of visual neglect , 1973, Neurology.

[58]  L. Johannsen,et al.  Time course of eye and head deviation in spatial neglect. , 2008, Neuropsychology.

[59]  Robert H. Wurtz,et al.  Thalamic pathways for active vision , 2011, Trends in Cognitive Sciences.

[60]  John K. Tsotsos,et al.  A Computational Learning Theory of Active Object Recognition Under Uncertainty , 2012, International Journal of Computer Vision.

[61]  N. Geschwind Disconnexion syndromes in animals and man. I. , 1965, Brain : a journal of neurology.

[62]  H. Duffau,et al.  Direct Evidence for a Parietal-Frontal Pathway Subserving Spatial Awareness in Humans , 2005, Science.

[63]  Karl J. Friston,et al.  The Functional Anatomy of Time: What and When in the Brain , 2016, Trends in Cognitive Sciences.

[64]  David J. Freedman,et al.  A hierarchy of intrinsic timescales across primate cortex , 2014, Nature Neuroscience.

[65]  M. Catani,et al.  The rises and falls of disconnection syndromes. , 2005, Brain : a journal of neurology.

[66]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[67]  R. Gregory Perceptions as hypotheses. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[68]  Adeel Razi,et al.  Bayesian model reduction and empirical Bayes for group (DCM) studies , 2016, NeuroImage.

[69]  Karl J. Friston,et al.  EEG and MEG Data Analysis in SPM8 , 2011, Comput. Intell. Neurosci..

[70]  Markus Lappe,et al.  Visual Space Constructed by Saccade Motor Maps , 2016, Front. Hum. Neurosci..

[71]  J. Hohwy The sense of self in the phenomenology of agency and perception , 2007 .

[72]  Karl J. Friston,et al.  Uncertainty, epistemics and active inference , 2017, Journal of The Royal Society Interface.

[73]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[74]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[75]  Biyu J. He,et al.  Breakdown of Functional Connectivity in Frontoparietal Networks Underlies Behavioral Deficits in Spatial Neglect , 2007, Neuron.

[76]  Vijay Kannan,et al.  Neural Substrates of Visuospatial Processing in Distinct Reference Frames: Evidence from Unilateral Spatial Neglect , 2009, Journal of Cognitive Neuroscience.

[77]  Paolo Bartolomeo,et al.  Dorsal and Ventral Parietal Contributions to Spatial Orienting in the Human Brain , 2011, The Journal of Neuroscience.

[78]  W. Fries Cortical projections to the superior colliculus in the macaque monkey: A retrograde study using horseradish peroxidase , 1984, The Journal of comparative neurology.

[79]  Karl J. Friston,et al.  Human visual exploration reduces uncertainty about the sensed world , 2018, PloS one.

[80]  Andrew B. Leber,et al.  Coordination of Voluntary and Stimulus-Driven Attentional Control in Human Cortex , 2005, Psychological science.

[81]  D. Lindley On a Measure of the Information Provided by an Experiment , 1956 .

[82]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.