If neuroimaging is the answer, what is the question?

It is unclear that we will come to a better understanding of mental processes simply by observing which neural loci are activated while subjects perform a task. Rather, I suggest here that it is better to come armed with a question that directs one to design tasks in ways that take advantage of the strengths of neuroimaging techniques (particularly positron emission tomography and functional magnetic resonance imaging). Here I develop a taxonomy of types of questions that can be easily addressed by such techniques. The first class of questions focuses on how information processing is implemented in the brain; these questions can be posed at a very coarse scale, focusing on the entire system that confers a particular ability, or at increasingly more specific scales, ultimately focusing on individual structures or processes. The second class of questions focuses on specifying when particular processes and structures are invoked; these questions focus on how one can use patterns of activation to infer that specific processes and structures were invoked, and on how processing changes in different circumstances. The use of neuroimaging to address these questions is illustrated with results from experiments on visual cognition, and caveats regarding the logic of inference in each case are noted. Finally, the necessary interplay between neuroimaging and behavioural studies is stressed.

[1]  A. Luria Higher Cortical Functions in Man , 1980, Springer US.

[2]  S. Kosslyn,et al.  Transcranial Magnetic Stimulation of Primary Motor Cortex Affects Mental Rotation , 2022 .

[3]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[4]  M. Posner,et al.  The attention system of the human brain. , 1990, Annual review of neuroscience.

[5]  Stephen M. Kosslyn,et al.  Pictures and names: Making the connection , 1984, Cognitive Psychology.

[6]  D. Ts'o,et al.  Functional organization of primate visual cortex revealed by high resolution optical imaging. , 1990, Science.

[7]  S. Kosslyn,et al.  Varieties of size-specific visual selection. , 1989 .

[8]  M. Denis,et al.  Reopening the Mental Imagery Debate: Lessons from Functional Anatomy , 1998, NeuroImage.

[9]  R. Weller Two cortical visual systems in Old World and New World primates. , 1988, Progress in brain research.

[10]  S. Kosslyn,et al.  Varieties of size-specific visual selection. , 1989, Journal of experimental psychology. General.

[11]  S. Petersen,et al.  The effects of practice on the functional anatomy of task performance. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Stephen M Kosslyn,et al.  Sequential processes in image generation , 1988, Cognitive Psychology.

[13]  S. Kosslyn Aspects of a cognitive neuroscience of mental imagery. , 1988, Science.

[14]  T. Paus Location and function of the human frontal eye-field: A selective review , 1996, Neuropsychologia.

[15]  P. Goldman-Rakic Topography of cognition: parallel distributed networks in primate association cortex. , 1988, Annual review of neuroscience.

[16]  S. Kosslyn,et al.  Individual Differences in Cerebral Blood Flow in Area 17 Predict the Time to Evaluate Visualized Letters , 1996, Journal of Cognitive Neuroscience.

[17]  A. Damasio,et al.  The frontal lobes , 1993 .

[18]  A Berthoz,et al.  Functional Neuroanatomy of the Human Visual Fixation System , 1995, The European journal of neuroscience.

[19]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[20]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[21]  S. Kosslyn,et al.  Identifying objects at different levels of hierarchy: A positron emission tomography study , 1995 .

[22]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[23]  L M Parsons,et al.  Imagined spatial transformation of one's body. , 1987, Journal of experimental psychology. General.

[24]  Zenon W. Pylyshyn,et al.  The Imagery Debate: Analogue Media Versus Tacit Knowledge , 1988 .

[25]  R. Shepard,et al.  Functional representations common to visual perception and imagination. , 1978, Journal of experimental psychology. Human perception and performance.

[26]  P. Goldman-Rakic,et al.  Dissociation of object and spatial processing domains in primate prefrontal cortex. , 1993, Science.

[27]  S. Kosslyn,et al.  Motor processes in mental rotation , 1998, Cognition.

[28]  Karl J. Friston,et al.  The colour centre in the cerebral cortex of man , 1989, Nature.

[29]  Leslie G. Ungerleider,et al.  Dissociation of object and spatial visual processing pathways in human extrastriate cortex. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[30]  S M Kosslyn,et al.  Identifying objects seen from different viewpoints. A PET investigation. , 1994, Brain : a journal of neurology.

[31]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[32]  G A Miller,et al.  Emotional imagery: conceptual structure and pattern of somato-visceral response. , 1980, Psychophysiology.

[33]  S. Kosslyn,et al.  Topographical representations of mental images in primary visual cortex , 1995, Nature.

[34]  S. Kosslyn,et al.  Wet Mind: The New Cognitive Neuroscience , 1995 .

[35]  L. Parsons Imagined spatial transformations of one's hands and feet , 1987, Cognitive Psychology.

[36]  S. Houle,et al.  Activation of medial temporal structures during episodic memory retrieval , 1996, Nature.

[37]  S. Kosslyn Image and brain: the resolution of the imagery debate , 1994 .

[38]  James L. McClelland,et al.  Psychological and biological models , 1986 .

[39]  Kim Sterelny,et al.  The Imagery Debate , 1991 .

[40]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[41]  Stephen M. Kosslyn,et al.  Neural Systems Activated during Visual Mental Imagery: A Review and Meta-Analyses , 2000 .

[42]  Karl J. Friston,et al.  Cognitive Conjunction: A New Approach to Brain Activation Experiments , 1997, NeuroImage.

[43]  S. Kosslyn,et al.  The role of area 17 in visual imagery: convergent evidence from PET and rTMS. , 1999, Science.

[44]  S. Kosslyn Image and mind , 1982 .

[45]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[46]  S. Kosslyn,et al.  Mental rotation of objects versus hands: neural mechanisms revealed by positron emission tomography. , 1998, Psychophysiology.

[47]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[48]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[49]  J L McGaugh,et al.  Amygdala activity at encoding correlated with long-term, free recall of emotional information. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[50]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[51]  K. Sterelny The Imagery Debate , 1986, Philosophy of Science.

[52]  S. Kosslyn,et al.  Neural effects of visualizing and perceiving aversive stimuli: a PET investigation. , 1996, Neuroreport.

[53]  S. Kosslyn,et al.  Neural Systems Shared by Visual Imagery and Visual Perception: A Positron Emission Tomography Study , 1997, NeuroImage.

[54]  J. Allman,et al.  Mapping human visual cortex with positron emission tomography , 1986, Nature.

[55]  Edward E. Smith,et al.  Categories and concepts , 1984 .

[56]  S. Petersen,et al.  Practice-related changes in human brain functional anatomy during nonmotor learning. , 1994, Cerebral cortex.

[57]  M Fujiki,et al.  [Transcranial magnetic stimulation]. , 2001, No shinkei geka. Neurological surgery.