Length and orientation constancy learning in 2-dimensions with auditory sensory substitution: the importance of self-initiated movement

A subset of sensory substitution (SS) devices translate images into sounds in real time using a portable computer, camera, and headphones. Perceptual constancy is the key to understanding both functional and phenomenological aspects of perception with SS. In particular, constancies enable object externalization, which is critical to the performance of daily tasks such as obstacle avoidance and locating dropped objects. In order to improve daily task performance by the blind, and determine if constancies can be learned with SS, we trained blind (N = 4) and sighted (N = 10) individuals on length and orientation constancy tasks for 8 days at about 1 h per day with an auditory SS device. We found that blind and sighted performance at the constancy tasks significantly improved, and attained constancy performance that was above chance. Furthermore, dynamic interactions with stimuli were critical to constancy learning with the SS device. In particular, improved task learning significantly correlated with the number of spontaneous left-right head-tilting movements while learning length constancy. The improvement from previous head-tilting trials even transferred to a no-head-tilt condition. Therefore, not only can SS learning be improved by encouraging head movement while learning, but head movement may also play an important role in learning constancies in the sighted. In addition, the learning of constancies by the blind and sighted with SS provides evidence that SS may be able to restore vision-like functionality to the blind in daily tasks.

[1]  Amir Amedi,et al.  Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind , 2012, Neuron.

[2]  Jamie Ward,et al.  Seeing with Sound? Exploring Different Characteristics of a Visual-to-Auditory Sensory Substitution Device , 2011, Perception.

[3]  C. Veraart,et al.  Functional Cerebral Reorganization for Auditory Spatial Processing and Auditory Substitution of Vision in Early Blind Subjects , 2006 .

[4]  Laurent Renier,et al.  Vertical-horizontal illusion present for sighted but not early blind humans using auditory substitution of vision , 2006, Perception & psychophysics.

[5]  M Critchley,et al.  Brain Mechanisms for Directed Attention , 1978, Journal of the Royal Society of Medicine.

[6]  Hervé Segond,et al.  Human Spatial Navigation via a Visuo-Tactile Sensory Substitution System , 2005, Perception.

[7]  J S Tittle,et al.  The visual perception of three-dimensional length. , 1996, Journal of experimental psychology. Human perception and performance.

[8]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[9]  H. A. Witkin,et al.  Studies in space orientation; perception of the upright in the absence of a visual field. , 1948, Journal of experimental psychology.

[10]  I. Biederman,et al.  Shape constancy and a perceptual bias towards symmetry , 1976 .

[11]  Day Rh,et al.  Mechanisms involved in visual orientation constancy. , 1969 .

[12]  Jaakko Hintikka,et al.  On the Logic of Perception , 1969 .

[13]  B Hughes,et al.  Perceptual learning of spatiotemporal events: evidence from an unfamiliar modality. , 1989, Journal of experimental psychology. Human perception and performance.

[14]  R Bruyer,et al.  The Ponzo Illusion with Auditory Substitution of Vision in Sighted and Early-Blind Subjects , 2005, Perception.

[15]  W. Epstein,et al.  Shape constancy: Functional relationships and theoretical formulations. , 1963 .

[16]  William H Warren,et al.  Distal Attribution and Distance Perception in Sensory Substitution , 2010, Perception.

[17]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[18]  I. Rock The Logic of Perception , 1983 .

[19]  Shachar Maidenbaum,et al.  Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution , 2012, Scientific Reports.

[20]  R H Day,et al.  Mechanisms involved in visual orientation constancy. , 1969, Psychological bulletin.

[21]  Peter B. L. Meijer,et al.  Functional recruitment of visual cortex for sound encoded object identification in the blind , 2009, Neuroreport.

[22]  R. Hetherington The Perception of the Visual World , 1952 .

[23]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[24]  A. Volder,et al.  Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects , 2007, Neuropsychologia.

[25]  Michael J. Proulx,et al.  The role of visual experience for the neural basis of spatial cognition , 2012, Neuroscience & Biobehavioral Reviews.

[26]  Ryota Kanai,et al.  Perceiving the Present and a Systematization of Illusions , 2008, Cogn. Sci..

[27]  Kristoffer Hougaard Madsen,et al.  Neural correlates of virtual route recognition in congenital blindness , 2010, Proceedings of the National Academy of Sciences.

[28]  J. Kevin O'Regan,et al.  What it is like to see: A sensorimotor theory of perceptual experience , 2001, Synthese.

[29]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[30]  Peter B. L. Meijer,et al.  Multisensory perceptual learning and sensory substitution , 2014, Neuroscience & Biobehavioral Reviews.

[31]  A. D. De Volder,et al.  Vision substitution and depth perception: Early blind subjects experience visual perspective through their ears , 2010, Disability and rehabilitation. Assistive technology.

[32]  Thomas D. Wright,et al.  Neuroscience and Biobehavioral Reviews Sensory Substitution as an Artificially Acquired Synaesthesia , 2022 .

[33]  R. Held,et al.  MOVEMENT-PRODUCED STIMULATION IN THE DEVELOPMENT OF VISUALLY GUIDED BEHAVIOR. , 1963, Journal of comparative and physiological psychology.

[34]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[35]  Andrew J. R. Simpson,et al.  Visual objects in the auditory system in sensory substitution: how much information do we need? , 2014, Multisensory research.

[36]  Elizabeth Neal,et al.  Visual Localization of the Vertical , 1926 .

[37]  J. O'Regan,et al.  Learning to Perceive with a Visuo — Auditory Substitution System: Localisation and Object Recognition with ‘The Voice’ , 2007, Perception.

[38]  C. Veraart,et al.  Assessment of sensory substitution prosthesis potentialities in minimalist conditions of learning , 2006 .

[39]  P. Stoerig,et al.  Seeing ‘Where’ through the Ears: Effects of Learning-by-Doing and Long-Term Sensory Deprivation on Localization Based on Image-to-Sound Substitution , 2008, PloS one.

[40]  Michael J. Proulx,et al.  How well do you see what you hear? The acuity of visual-to-auditory sensory substitution , 2013, Front. Psychol..

[41]  M. Proulx,et al.  Increased Signal Complexity Improves the Breadth of Generalization in Auditory Perceptual Learning , 2013, Neural plasticity.

[42]  Christian Scheiber,et al.  What neuroimaging tells us about sensory substitution , 2007, Neuroscience & Biobehavioral Reviews.

[43]  P. Bach-y-Rita,et al.  Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note. , 1998, Journal of rehabilitation research and development.