Deep-BCN: Deep Networks Meet Biased Competition to Create a Brain-Inspired Model of Attention Control

The mechanism of attention control is best described by biased-competition theory (BCT), which suggests that a top-down goal state biases a competition among object representations for the selective routing of a visual input for classification. Our work advances this theory by making it computationally explicit as a deep neural network (DNN) model, thereby enabling predictions of goal-directed attention control using real-world stimuli. This model, which we call Deep-BCN, is built on top of an 8-layer DNN pre-trained for object classification, but has layers mapped to early visual (V1, V2/V3, V4), ventral (PIT, AIT), and frontal (PFC) brain areas that have their functional connectivity informed by BCT. Deep-BCN also has a superior colliculus and a frontal-eye field, and can therefore make eye movements. We compared Deep-BCN's eye movements to those made from 15 people performing a categorical search for one of 25 target object categories, and found that it predicted both the number of fixations during search and the saccade-distance travelled before search termination. With Deep-BCN a DNN implementation of BCT now exists, which can be used to predict the neural and behavioral responses of an attention control mechanism as it mediates a goal-directed behavior-in our study the eye movements made in search of a target goal.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[3]  R. Desimone,et al.  Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. , 1997, Journal of neurophysiology.

[4]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[5]  C. Gilbert,et al.  Top-down influences on visual processing , 2013, Nature Reviews Neuroscience.

[6]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[7]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[8]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[9]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[10]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[11]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[12]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[13]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[15]  R. Desimone,et al.  Neural Mechanisms of Object-Based Attention , 2014, Science.

[16]  Yifan Peng,et al.  Modelling eye movements in a categorical search task , 2013, Philosophical Transactions of the Royal Society B: Biological Sciences.

[17]  P. Roelfsema,et al.  Different States in Visual Working Memory: When It Guides Attention and When It Does Not , 2022 .

[18]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[19]  G. Deco,et al.  Top-down selective visual attention: A neurodynamical approach , 2001 .

[20]  Luiz Pessoa,et al.  What and where pathways , 2008, Scholarpedia.

[21]  F. Hamker.  A dynamic model of how feature cues guide spatial attention , 2004, Vision Research.

[22]  R. Desimone,et al.  Attention Increases Sensitivity of V4 Neurons , 2000, Neuron.

[23]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[24]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[25]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[26]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[27]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[28]  Robert Desimone,et al.  Parallel and Serial Neural Mechanisms for Visual Search in Macaque Area V4 , 2005, Science.

[29]  J. Duncan.  Cooperating brain systems in selective perception and action. , 1996 .

[30]  D. Heeger,et al.  Sustained Activity in Topographic Areas of Human Posterior Parietal Cortex during Memory-Guided Saccades , 2006, The Journal of Neuroscience.

[31]  Leslie G. Ungerleider.  Two cortical visual systems , 1982 .

[32]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[33]  Gregory J. Zelinsky,et al.  A Model of the Superior Colliculus Predicts Fixation Locations during Scene Viewing and Visual Search , 2017, The Journal of Neuroscience.

[34]  Antonio Torralba,et al.  Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition , 2016, ArXiv.

[35]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.

[36]  Sabine Kastner,et al.  Topographic maps in human frontal cortex revealed in memory-guided saccade and spatial working-memory tasks. , 2007, Journal of neurophysiology.

[37]  R. Desimone,et al.  High-Frequency, Long-Range Coupling Between Prefrontal and Visual Cortex During Attention , 2009, Science.

[38]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[39]  G. Boynton,et al.  Feature-Based Attentional Modulations in the Absence of Direct Visual Stimulation , 2007, Neuron.

[40]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[41]  C. Bundesen.  A theory of visual attention. , 1990, Psychological review.

[42]  Leslie G. Ungerleider,et al.  ‘What’ and ‘where’ in the human brain , 1994, Current Opinion in Neurobiology.

[43]  N. P. Bichot,et al.  A Source for Feature-Based Attention in the Prefrontal Cortex , 2015, Neuron.

[44]  G. Glover,et al.  Retinotopic organization in human visual cortex and the spatial precision of functional MRI. , 1997, Cerebral cortex.

[45]  C. Bundesen,et al.  A neural theory of visual attention and short-term memory (NTVA) , 2011, Neuropsychologia.

[46]  G. Boynton,et al.  Global effects of feature-based attention in human visual cortex , 2002, Nature Neuroscience.

[47]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[48]  Diane M. Beck,et al.  Top-down and bottom-up mechanisms in biasing competition in the human brain , 2009, Vision Research.

[49]  Timothy F. Brady,et al.  Conceptual Distinctiveness Supports Detailed Visual Long-term Memory for Real-world Objects the Fidelity of Long-term Memory for Visual Information , 2022 .

[50]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[51]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[52]  Robert Desimone,et al.  Feature-Based Attention in the Frontal Eye Field and Area V4 during Visual Search , 2011, Neuron.

[53]  John K. Tsotsos,et al.  Computational models of visual attention , 2011, Vision Research.

[54]  John Duncan,et al.  Shape-specific preparatory activity mediates attention to targets in human visual cortex , 2009, Proceedings of the National Academy of Sciences.

[55]  Hans-Jochen Heinze,et al.  Object-based attention involves the sequential activation of feature-specific cortical modules , 2014, Nature Neuroscience.

[56]  Ha Hong,et al.  Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream , 2013, NIPS.

[57]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.

[58]  Koray Kavukcuoglu,et al.  Visual Attention , 2020, Computational Models for Cognitive Vision.

[59]  Dwight J. Kravitz,et al.  The ventral visual pathway: an expanded neural framework for the processing of object quality , 2013, Trends in Cognitive Sciences.

[60]  F. Tong,et al.  Neural mechanisms of object-based attention. , 2015, Cerebral cortex.

[61]  R. Desimone,et al.  Interacting Roles of Attention and Visual Salience in V4 , 2003, Neuron.

[62]  C. Bundesen,et al.  A neural theory of visual attention: bridging cognition and neurophysiology. , 2005, Psychological review.

[63]  J. Findlay,et al.  The Relationship between Eye Movements and Spatial Attention , 1986, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[64]  John H. R. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[65]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[66]  Leslie G. Ungerleider,et al.  Texture segregation in the human visual cortex: A functional MRI study. , 2000, Journal of neurophysiology.

[67]  F. Hamker.  The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement. , 2005, Cerebral cortex.

[68]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[69]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..