Serial exploration of faces: comparing vision and touch.

Even though we can recognize faces by touch surprisingly well, haptic face recognition performance is still worse than for visual exploration. One possibility for this performance difference might be due to different encoding strategies in the two modalities, namely, holistic encoding in vision versus serial encoding in haptics. Here, we tested this hypothesis by promoting serial encoding in vision, using a novel, gaze-restricted display that limited the effective field of view in vision to resemble that of haptic exploration. First, we compared haptic with gaze-restricted and unrestricted visual face recognition. Second, we used the face inversion paradigm to assess how encoding differences might affect processing strategies (featural vs. holistic). By promoting serial encoding in vision, we found equal face recognition performance in vision and haptics with a clear switch from holistic to featural processing, suggesting that performance differences in visual and haptic face recognition are due to modality-specific encoding strategies.

[1]  Sasan Mahmoodi,et al.  How do monkeys view faces?—a study of eye movements , 2003, Experimental Brain Research.

[2]  R. Joseph,et al.  Holistic and part-based face recognition in children with autism. , 2003, Journal of child psychology and psychiatry, and allied disciplines.

[3]  Jason J S Barton,et al.  Information Processing during Face Recognition: The Effects of Familiarity, Inversion, and Morphing on Scanning Fixations , 2006, Perception.

[4]  V. Bruce,et al.  The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology When Inverted Faces Are Recognized: the Role of Configural Information in Face Recognition , 2022 .

[5]  J. Tanaka,et al.  Features and their configuration in face recognition , 1997, Memory & cognition.

[6]  John M Henderson,et al.  The face inversion effect is not a consequence of aberrant eye movements , 2007, Memory & cognition.

[7]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[8]  G. Schwarzer,et al.  Development of face processing: the effect of face inversion. , 2000, Child development.

[9]  A. Treisman.  Contextual Cues in Selective Listening , 1960 .

[10]  S. Lederman,et al.  Face recognition by hand , 2002, Perception & psychophysics.

[11]  J. Bartlett,et al.  Inversion and processing of component and spatial-relational information in faces. , 1996, Journal of experimental psychology. Human perception and performance.

[12]  P. Cavanagh,et al.  The Capacity of Visual Short-Term Memory is Set Both by Visual Information Load and by Number of Objects , 2004, Psychological science.

[13]  D. Maurer,et al.  The many faces of configural processing , 2002, Trends in Cognitive Sciences.

[14]  D. Hay,et al.  Developmental changes in the recognition of faces and facial features. , 2000 .

[15]  Susan J. Lederman,et al.  Haptic Classification of Facial Identity in 2D Displays: Configural versus Feature-Based Processing , 2010, IEEE Transactions on Haptics.

[16]  C. Wallraven,et al.  Processing of facial identity and expression: a psychophysical, physiological, and computational perspective. , 2006, Progress in brain research.

[17]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[18]  G. Rhodes,et al.  Holistic Processing of Faces in Preschool Children and Adults , 2003, Psychological science.

[19]  K. Verfaillie,et al.  Face inversion impairs holistic perception: evidence from gaze-contingent stimulation. , 2010, Journal of vision.

[20]  M. Ikeda,et al.  Integrating time for visual pattern perception and a comparison with the tactile mode , 1978, Vision Research.

[21]  S. Carey,et al.  Why faces are and are not special: an effect of expertise. , 1986 .

[22]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.

[23]  Kris McDaniel,et al.  Parts and Wholes , 2010 .

[24]  A. G. Goldstein,et al.  Recognition of human faces from isolated facial features: A developmental study , 1966 .

[25]  Susan J. Lederman,et al.  Haptic Face Processing and Its Relation to Vision , 2010 .

[26]  S. Carey,et al.  Why faces are and are not special: an effect of expertise. , 1986, Journal of experimental psychology. General.

[27]  Heinrich H. Bülthoff,et al.  Cross-Modal Transfer in Visual and Haptic Face Recognition , 2009, IEEE Transactions on Haptics.

[28]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2004, Perception & Psychophysics.

[29]  Gyula Kovács,et al.  Inverted Faces , 2003, Perception.

[30]  Emiliano Ricciardi,et al.  Beyond sensory images: Object-based representation in the human ventral pathway. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[31]  S. Lederman,et al.  TACTUAL PERCEPTION , 2003 .

[32]  R. Klatzky,et al.  Similarity of Tactual and Visual Picture Recognition with Limited Field of View , 1991, Perception.

[33]  Christoph D. Dahl,et al.  Humans and Macaques Employ Similar Face-Processing Strategies , 2009, Current Biology.

[34]  M. Tarr,et al.  Becoming a “Greeble” Expert: Exploring Mechanisms for Face Recognition , 1997, Vision Research.

[35]  R. Desimone,et al.  Neural mechanisms for visual memory and their role in attention. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[36]  J. Sergent.  An investigation into component and configural processes underlying face perception. , 1984, British journal of psychology.

[37]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[38]  H. Pick,et al.  Intersensory Perception and Sensory Integration , 1981 .

[39]  S. Carey,et al.  From piecemeal to configurational representation of faces. , 1977, Science.

[40]  D. Maurer,et al.  Developmental changes in face processing skills. , 2003, Journal of experimental child psychology.

[41]  G. J. Walker-Smith,et al.  The effects of delay and exposure duration in a face recognition task , 1978 .

[42]  S. Lederman,et al.  A Haptic Face-Inversion Effect , 2006, Perception.

[43]  J. Bartlett,et al.  Inversion and processing of component and spatial–relational information in faces. , 1996 .

[44]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[45]  S. McKelvie,et al.  The Role of Eyes and Mouth in the Memory of a Face. , 1976 .

[46]  H. Bülthoff,et al.  Multimodal similarity and categorization of novel, three-dimensional objects , 2007, Neuropsychologia.

[47]  S. Knecht,et al.  Parallel and serial processing of haptic information in man: Effects of parietal lesions on sensorimotor hand function , 1996, Neuropsychologia.

[48]  P. Schyns,et al.  Show Me the Features! Understanding Recognition From the Use of Visual Information , 2002, Psychological science.

[49]  L. Squire.  Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory , 1992, Journal of Cognitive Neuroscience.

[50]  Heikki Hämäläinen,et al.  Different working memory capacity in normal young adults for visual and tactile letter recognition task. , 2005, Scandinavian journal of psychology.

[51]  S. Lederman,et al.  Haptic face recognition and prosopagnosia , 2004, Neuropsychologia.

[52]  Mark H. Johnson,et al.  Eye contact detection in humans from birth , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Susanna Millar,et al.  External and body-centered frames of reference in spatial memory: Evidence from touch , 2004, Perception & psychophysics.

[54]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[55]  P. Bennett,et al.  Inversion Leads to Quantitative, Not Qualitative, Changes in Face Processing , 2004, Current Biology.

[56]  G. Woodman,et al.  Storage of features, conjunctions and objects in visual working memory. , 2001, Journal of experimental psychology. Human perception and performance.

[57]  S Lakatos,et al.  Haptic form perception: Relative salience of local and global features , 1999, Perception & psychophysics.

[58]  T. Valentine.  Upside-down faces: a review of the effect of inversion upon face recognition. , 1988, British journal of psychology.

[59]  Michael G. Strintzis,et al.  Face Recognition , 2008, Encyclopedia of Multimedia.

[60]  R. Yin.  Looking at Upside-down Faces , 1969 .

[61]  H. Bülthoff,et al.  Visual and haptic perceptual spaces show high similarity in humans. , 2010, Journal of vision.

[62]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[63]  F. Newell,et al.  Are representations of unfamiliar faces independent of encoding modality? , 2007, Neuropsychologia.

[64]  Isabel Gauthier,et al.  Perceptual Expertise Effects Are Not All or None: Spatially Limited Perceptual Expertise for Faces in a Case of Prosopagnosia , 2006, Journal of Cognitive Neuroscience.

[65]  T. Langdell,et al.  Recognition of faces: an approach to the study of autism. , 1978, Journal of child psychology and psychiatry, and allied disciplines.

[66]  H. Bülthoff,et al.  Face recognition under varying poses: The role of texture and shape , 1996, Vision Research.

[67]  N. Haig.  Exploring Recognition with Interchanged Facial Features , 1986, Perception.

[68]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[69]  Christoph D. Dahl,et al.  Individuation and holistic processing of faces in rhesus monkeys , 2007, Proceedings of the Royal Society B: Biological Sciences.

[70]  R. Caldara,et al.  Inverting Faces Does Not Abolish Cultural Diversity in Eye Movements , 2010, Perception.