Action observation: the less-explored part of higher-order vision

Little is presently known about action observation, an important perceptual component of high-level vision. To investigate this aspect of perception, we introduce a two-alternative forced-choice task for observed manipulative actions while varying duration or signal strength by noise injection. We show that accuracy and reaction time in this task can be modeled by a diffusion process for different pairs of action exemplars. Furthermore, discrimination of observed actions is largely viewpoint-independent, cannot be reduced to judgments about the basic components of action: shape and local motion, and requires a minimum duration of about 150–200 ms. These results confirm that action observation is a distinct high-level aspect of visual perception based on temporal integration of visual input generated by moving body parts. This temporal integration distinguishes it from object or scene perception, which require only very brief presentations and are viewpoint-dependent. The applicability of a diffusion model suggests that these aspects of high-level vision differ mainly at the level of the sensory neurons feeding the decision processes.

[1]  A. Voss,et al.  Diffusion models in experimental psychology: a practical introduction. , 2013, Experimental psychology.

[2]  N. Kanwisher,et al.  Activation in Human MT/MST by Static Images with Implied Motion , 2000, Journal of Cognitive Neuroscience.

[3]  Timothy D. Hanks,et al.  Microstimulation of macaque area LIP affects decision-making in a motion discrimination task , 2006, Nature Neuroscience.

[4]  Luca Turella,et al.  MEG Multivariate Analysis Reveals Early Abstract Action Representations in the Lateral Occipitotemporal Cortex , 2015, The Journal of Neuroscience.

[5]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[6]  E. Bullmore,et al.  The functional neuroanatomy of implicit-motion perception or ‘representational momentum’ , 2000, Current Biology.

[7]  Leslie G. Ungerleider,et al.  A general mechanism for perceptual decision-making in the human brain , 2004, Nature.

[8]  M. Corbetta,et al.  Sensory-motor mechanisms in human parietal cortex underlie arbitrary visual decisions , 2008, Nature Neuroscience.

[9]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[10]  H. Bülthoff,et al.  View dependencies in the visual recognition of social interactions , 2013, Front. Psychol..

[11]  D. Sheinberg,et al.  Temporal Cortex Neurons Encode Articulated Actions as Slow Sequences of Integrated Poses , 2010, The Journal of Neuroscience.

[12]  Guy A. Orban,et al.  The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study , 2015, Human brain mapping.

[13]  N. Troje,et al.  The Inversion Effect in Biological Motion Perception: Evidence for a “Life Detector”? , 2006, Current Biology.

[14]  Alice C. Roy,et al.  Encoding of human action in Broca's area. , 2009, Brain : a journal of neurology.

[15]  G. Orban,et al.  Human orientation discrimination tested with long stimuli , 1984, Vision Research.

[16]  M. Shadlen,et al.  Microstimulation of visual cortex affects the speed of perceptual decisions , 2003, Nature Neuroscience.

[17]  Heinrich H Bülthoff,et al.  Image-based object recognition in man, monkey and machine , 1998, Cognition.

[18]  R. Vogels,et al.  Functional differentiation of macaque visual temporal cortical neurons using a parametric action space. , 2009, Cerebral cortex.

[19]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[20]  M. Shadlen,et al.  The effect of stimulus strength on the speed and accuracy of a perceptual decision. , 2005, Journal of vision.

[21]  H. Bülthoff,et al.  Action Recognition and Movement Direction Discrimination Tasks Are Associated with Different Adaptation Patterns , 2016, Front. Hum. Neurosci..

[22]  S. McKee.  A local mechanism for differential velocity detection , 1981, Vision Research.

[23]  E. Adelson,et al.  Accuracy and speed of material categorization in real-world images. , 2014, Journal of vision.

[24]  S. Thorpe,et al.  The time course of visual processing: Backward masking and natural scene categorisation , 2005, Vision Research.

[25]  R. Blake,et al.  Perception of coherent motion, biological motion and form-from-motion under dim-light conditions , 1999, Vision Research.

[26]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[27]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[28]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[29]  Markus Lappe,et al.  The role of spatial and temporal information in biological motion perception , 2008, Advances in cognitive psychology.

[30]  G. Orban,et al.  Common and segregated processing of observed actions in human SPL. , 2013, Cerebral cortex.

[31]  S. Edelman,et al.  Orientation dependence in the recognition of familiar and novel views of three-dimensional objects , 1992, Vision Research.

[32]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[33]  A. Caramazza,et al.  Typical action perception and interpretation without motor simulation , 2015, Proceedings of the National Academy of Sciences.

[34]  Cosimo Urgesi,et al.  Compensatory plasticity in the action observation network: virtual lesions of STS enhance anticipatory simulation of seen actions. , 2013, Cerebral cortex.

[35]  Thomas Wolf,et al.  Continuous Theta-Burst Stimulation Demonstrates a Causal Role of Premotor Homunculus in Action Understanding , 2014, Psychological science.

[36]  D I Perrett,et al.  Frameworks of analysis for the neural representation of animate objects and actions. , 1989, The Journal of experimental biology.

[37]  Wei-Song Lin,et al.  A computational visual saliency model based on statistics and machine learning. , 2014, Journal of vision.

[38]  G Kovács,et al.  Cortical correlate of pattern backward masking. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[39]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[40]  M. Tovée,et al.  The responses of neurons in the temporal cortex of primates, and face identification and detection , 1994, Experimental Brain Research.

[41]  M. Candidi,et al.  Representation of body identity and body actions in extrastriate body area and ventral premotor cortex , 2007, Nature Neuroscience.

[42]  G. Rizzolatti,et al.  Stereoscopically Observing Manipulative Actions , 2016, Cerebral cortex.

[43]  V. Caggiano,et al.  Physiologically Inspired Model for the Visual Recognition of Transitive Hand Actions , 2013, The Journal of Neuroscience.

[44]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[45]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[46]  Andreas Voss,et al.  A diffusion model analysis of adult age differences in episodic and semantic long-term memory retrieval. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[47]  W. Prinz.  Perception and Action Planning , 1997 .

[48]  Luciano Fadiga,et al.  Role of Broca's area in encoding sequential human actions: a virtual lesion study , 2009, Neuroreport.

[49]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[50]  Eric-Jan Wagenmakers,et al.  Methodological and empirical developments for the Ratcliff diffusion model of response times and accuracy , 2009 .

[51]  Emily S. Cross,et al.  Sensitivity of the action observation network to physical and observational learning. , 2008, Cerebral cortex.

[52]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[53]  G. Rizzolatti,et al.  View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex , 2011, Current Biology.

[54]  R. Kiani,et al.  Microstimulation of inferotemporal cortex influences face categorization , 2006, Nature.

[55]  G. Rizzolatti,et al.  What and Why Understanding in Autism Spectrum Disorders and Williams Syndrome: Similarities and Differences , 2014, Autism research : official journal of the International Society for Autism Research.

[56]  R. Romo,et al.  Correlated Neuronal Discharges that Increase Coding Efficiency during Perceptual Discrimination , 2003, Neuron.

[57]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[58]  T. McNamara,et al.  Viewpoint Dependence in Scene Recognition , 1997 .

[59]  W. Dittrich.  Action Categories and the Perception of Biological Motion , 1993, Perception.

[60]  Marc M. Van Hulle,et al.  Optic flow from unstable sequences through local velocity constancy maximization , 2009, Image Vis. Comput..

[61]  Jeannette A. M. Lorteije,et al.  Implied Motion Activation in Cortical Area MT Can Be Explained by Visual Low-level Features , 2011, Journal of Cognitive Neuroscience.

[62]  G. Orban,et al.  Coding observed motor acts: different organizational principles in the parietal and premotor cortex of humans. , 2010, Journal of neurophysiology.