Cognitive programs: software for attention's executive

What are the computational tasks that an executive controller for visual attention must solve? This question is posed in the context of the Selective Tuning model of attention. The range of required computations go beyond top-down bias signals or region-of-interest determinations, and must deal with overt and covert fixations, process timing and synchronization, information routing, memory, matching control to task, spatial localization, priming, and coordination of bottom-up with top-down information. During task execution, results must be monitored to ensure the expected results. This description includes the kinds of elements that are common in the control of any kind of complex machine or system. We seek a mechanistic integration of the above, in other words, algorithms that accomplish control. Such algorithms operate on representations, transforming a representation of one kind into another, which then forms the input to yet another algorithm. Cognitive Programs (CPs) are hypothesized to capture exactly such representational transformations via stepwise sequences of operations. CPs, an updated and modernized offspring of Ullman's Visual Routines, impose an algorithmic structure to the set of attentional functions and play a role in the overall shaping of attentional modulation of the visual system so that it provides its best performance. This requires that we consider the visual system as a dynamic, yet general-purpose processor tuned to the task and input of the moment. This differs dramatically from the almost universal cognitive and computational views, which regard vision as a passively observing module to which simple questions about percepts can be posed, regardless of task. Differing from Visual Routines, CPs explicitly involve the critical elements of Visual Task Executive (vTE), Visual Attention Executive (vAE), and Visual Working Memory (vWM). Cognitive Programs provide the software that directs the actions of the Selective Tuning model of visual attention.

[1]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[2]  L. Itti,et al.  Mechanisms of top-down attention , 2011, Trends in Neurosciences.

[3]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[4]  Victor A. F. Lamme,et al.  The implementation of visual routines , 2000, Vision Research.

[5]  Nikolaus Kriegeskorte,et al.  What's there, distinctly, when and where? , 2014, Nature Neuroscience.

[6]  Dana H. Ballard,et al.  Visual routines for autonomous driving , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  S. Ullman.  Visual routines , 1984, Cognition.

[8]  T. Shallice.  From Neuropsychology to Mental Structure: Converging Operations: Specific Syndromes and Evidence from Normal Subjects , 1988 .

[9]  John K. Tsotsos,et al.  Visual attention in dynamic environments , 2014 .

[10]  Viola S. Störmer,et al.  Feature-Based Attention Elicits Surround Suppression in Feature Space , 2014, Current Biology.

[11]  John K. Tsotsos,et al.  The center-surround profile of the focus of attention arises from recurrent processing in visual cortex. , 2009, Cerebral cortex.

[12]  M. Posner,et al.  Attention and cognitive control. , 1975 .

[13]  Søren Kyllingsbæk,et al.  The Theory of Visual Attention without the race: a new model of visual selection , 2012, CogSci.

[14]  John K. Tsotsos,et al.  Neurobiology of Attention , 2005 .

[15]  T Shallice,et al.  Dual functions of consciousness. , 1972, Psychological review.

[16]  John K. Tsotsos,et al.  The different stages of visual recognition need different attentional binding strategies , 2008, Brain Research.

[17]  P. Roelfsema.  Elemental operations in vision , 2005, Trends in Cognitive Sciences.

[18]  James J. Clark,et al.  Modal Control Of An Attentive Vision System , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[19]  Rajesh P. N. Rao,et al.  Embodiment is the foundation, not a level , 1996, Behavioral and Brain Sciences.

[20]  Ned T. Sahin,et al.  Dynamic circuit motifs underlying rhythmic gain control, gating and integration , 2014, Nature Neuroscience.

[21]  M. Just,et al.  From the Selectedworks of Marcel Adam Just the Organization of Thinking: What Functional Brain Imaging Reveals about the Neuroarchitecture of Complex Cognition , 2022 .

[22]  Alan S. Brown,et al.  Information Processing and Cognition: The Loyola Symposium , 1976 .

[23]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[24]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[25]  John K. Tsotsos,et al.  Different Binding Strategies for the Different Stages of Visual Recognition , 2007, BVAI.

[26]  A. Stark,et al.  cis-Regulatory Requirements for Tissue-Specific Programs of the Circadian Clock , 2014, Current Biology.

[27]  John K. Tsotsos,et al.  Attentional Modulation and Selection – An Integrated Approach , 2014, PloS one.

[28]  Narayanan Srinivasan,et al.  The interplay of attention and consciousness in visual search, attentional blink and working memory consolidation , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[29]  Leslie G. Ungerleider,et al.  The prefrontal cortex and the executive control of attention , 2008, Experimental Brain Research.

[30]  安藤 広志,et al.  20世紀の名著名論：David Marr:Vision:a Computational Investigation into the Human Representation and Processing of Visual Information , 2005 .

[31]  Satyajit Rao,et al.  Visual routines and attention , 1998 .

[32]  Donald A. Norman,et al.  Attention to Action , 1986 .

[33]  S. Kyllingsbaek,et al.  Modeling visual attention. , 2006, Behavior research methods.

[34]  Ian Horswill,et al.  Visual Routines and Visual Search: A Real-Time Implementation and an Automata-Theoretic Analysis , 1995, IJCAI.

[35]  Kjell Brunnström,et al.  Active fixation for scene exploration , 1996, International Journal of Computer Vision.

[36]  John K. Tsotsos,et al.  It’s all about the constraints , 2014, Current Biology.

[37]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[38]  S. Ullman,et al.  Curve tracing: A possible basic operation in the perception of spatial relations , 1986, Memory & cognition.

[39]  John K. Tsotsos,et al.  Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Raymond Klein,et al.  Inhibition of return , 2000, Trends in Cognitive Sciences.

[41]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[42]  John K. Tsotsos.  Cognitive programs: towards an executive controller for visual attention , 2013 .

[43]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[44]  M. Posner,et al.  Attention, self-regulation and consciousness. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[45]  John K. Tsotsos,et al.  Attention links sensing to recognition , 2008, Image Vis. Comput..

[46]  P. Rabbitt,et al.  Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption , 1989 .

[47]  Pieter R Roelfsema,et al.  Subtask sequencing in the primary visual cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Ronald A. Rensink,et al.  Competition for consciousness among visual events: the psychophysics of reentrant visual processes. , 2000, Journal of experimental psychology. General.

[49]  C. Bundesen,et al.  A neural theory of visual attention: bridging cognition and neurophysiology. , 2005, Psychological review.

[50]  John K. Tsotsos.  A Computational Perspective on Visual Attention , 2011 .

[51]  M. Pinsk,et al.  Attention modulates responses in the human lateral geniculate nucleus , 2002, Nature Neuroscience.

[52]  L. Itti,et al.  A brief and selective history of attention , 2005 .

[53]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[54]  John K. Tsotsos.  Analyzing vision at the complexity level , 1990, Behavioral and Brain Sciences.

[55]  H. J. Muller,et al.  Reflexive and voluntary orienting of visual attention: time course of activation and resistance to interruption. , 1989, Journal of experimental psychology. Human perception and performance.

[56]  Hans-Jochen Heinze,et al.  The spatial profile of the focus of attention in visual search: Insights from MEG recordings , 2010, Vision Research.

[57]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[58]  J. Deutsch.  Perception and Communication , 1958, Nature.

[59]  Andrew Kachites McCallum,et al.  Learning Visual Routines with Reinforcement Learning , 1996 .

[60]  S. Kyllingsbæk.  Modeling visual attention , 2006 .

[61]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[62]  Sashank Varma,et al.  Criteria for the Design and Evaluation of Cognitive Architectures , 2011, Cogn. Sci..

[63]  M. Carrasco.  Visual attention: The past 25 years , 2011, Vision Research.

[64]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[65]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[66]  John K. Tsotsos.  The Complexity of Perceptual Search Tasks , 1989, IJCAI.

[67]  D. Ballard,et al.  Modelling the role of task in the control of gaze , 2009, Visual cognition.

[68]  Nikola T. Markov,et al.  Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex , 2013, The Journal of comparative neurology.

[69]  John K. Tsotsos,et al.  Attending to visual motion , 2005, Comput. Vis. Image Underst..

[70]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[71]  Dana H. Ballard,et al.  A Visual Control Architecture for a Virtual Humanoid , 2001 .

[72]  Earl K Miller,et al.  Cortical circuits for the control of attention , 2012, Current Opinion in Neurobiology.

[73]  Trevor Darrell,et al.  Evolving Visual Routines , 1994, Artificial Life.

[74]  Pieter R. Roelfsema,et al.  The Brain's Router: A Cortical Network Model of Serial Processing in the Primate Brain , 2010, PLoS Comput. Biol..

[75]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[76]  John K. Tsotsos,et al.  The Roles of Endstopped and Curvature Tuned Computations in a Hierarchical Representation of 2D Shape , 2012, PloS one.

[77]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[78]  J. Changeux,et al.  Experimental and Theoretical Approaches to Conscious Processing , 2011, Neuron.

[79]  Joshua D. Cosman,et al.  The Control of Visual Attention , 2014 .

[80]  John K. Tsotsos,et al.  Towards a Biologically Plausible Active Visual Search Model , 2004, WAPCV.

[81]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[82]  Sabine Kastner,et al.  The Oxford Handbook of Attention , 2014 .

[83]  John K. Tsotsos,et al.  Attention and Visual Search , 2007, Int. J. Neural Syst..