Intermodal recoding of a video game: Learning to process signals for motion perception in a pure auditory environment

The aim of the study reported here was to investigate people's ability to learn a new auditory environment. A specific coding was developed in order to design an auditory version of a vision-based simple video game involving moving objects. We investigated the learning process of participants in this purely auditory game and the influence of the following factors on performance: gender, level of practice in video games and three cognitive abilities, namely, mental rotation, visual attention and spatial perception. The results revealed that occasional and regular players' level of performance significantly improved with practice, whereas those of nonplayers did not. They also revealed a significant influence of gender. Scores in the mental rotation test was found to be correlated only with the final level of performance. This study provides insights into people's ability to learn new ways to perceive and interact with auditory objects in motion and highlights the cognitive abilities that influence this learning. Copyright © 2015John Wiley & Sons, Ltd.

[1]  Malika Auvray,et al.  The Vibe: a versatile vision-to-audition sensory substitution device , 2010 .

[2]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[3]  J. O'Regan,et al.  Learning to Perceive with a Visuo — Auditory Substitution System: Localisation and Object Recognition with ‘The Voice’ , 2007, Perception.

[4]  Ophelia Deroy,et al.  Reading the World through the Skin and Ears: A New Perspective on Sensory Substitution , 2012, Front. Psychology.

[5]  C S Green,et al.  Action-Video-Game Experience Alters the Spatial Resolution of Vision , 2007, Psychological science.

[6]  Amir Amedi,et al.  The neural network of sensory-substitution object shape recognition , 2011 .

[7]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[8]  C. S. Green,et al.  Enumeration versus multiple object tracking: the case of action video game players , 2006, Cognition.

[9]  S. Savel.  Individual differences and left/right asymmetries in auditory space perception. I. Localization of low-frequency sounds in free field , 2009, Hearing Research.

[10]  Amir Amedi,et al.  ‘Visual’ Acuity of the Congenitally Blind Using Visual-to-Auditory Sensory Substitution , 2012, PloS one.

[11]  C. S. Green,et al.  Changes in search rate but not in the dynamics of exogenous attention in action videogame players , 2011, Attention, perception & psychophysics.

[12]  C. S. Green,et al.  Action video game modifies visual selective attention , 2003, Nature.

[13]  J. Pratt,et al.  The effects of action video game experience on the time course of inhibition of return and the efficiency of visual search. , 2005, Acta psychologica.

[14]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[15]  Irwin Silverman,et al.  Sex differences in spatial abilities: Evolutionary theory and data. , 1992 .

[16]  U. Polat,et al.  Enhancing the contrast sensitivity function through action video game training , 2009, Nature Neuroscience.

[17]  Palvi Bhardwaj,et al.  Action video game players and deaf observers have larger Goldmann visual fields , 2010, Vision Research.

[18]  J. Pratt,et al.  Playing an Action Video Game Reduces Gender Differences in Spatial Cognition , 2007, Psychological science.

[19]  Fabien Scalzo,et al.  Reducing backward masking through action game training. , 2010, Journal of vision.

[20]  Shachar Maidenbaum,et al.  Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution , 2012, Scientific Reports.

[21]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[22]  C. S. Green,et al.  Brains on video games , 2011, Nature Reviews Neuroscience.

[23]  Susan D. Voyer,et al.  Magnitude of sex differences in spatial abilities: a meta-analysis and consideration of critical variables. , 1995, Psychological bulletin.

[24]  Malika Auvray,et al.  Perception With Compensatory Devices: From Sensory Substitution to Sensorimotor Extension , 2009, Cogn. Sci..

[25]  Daphne Bavelier,et al.  The effect of action video game experience on task-switching , 2012, Comput. Hum. Behav..

[26]  P. Frensch,et al.  Effects of video game playing on measures of spatial performance: Gender effects in late adolescence. , 1994 .

[27]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[28]  C S Green,et al.  Video games as a tool to train visual skills. , 2008, Restorative neurology and neuroscience.

[29]  C. Geiser,et al.  The relationship between computer-game preference, gender, and mental-rotation ability , 2006 .

[30]  Patricia A. McClurg,et al.  Computer Games: Environments for Developing Spatial Cognition? , 1987 .

[31]  B. Papsin,et al.  The effect of aging on horizontal plane sound localization. , 2000, The Journal of the Acoustical Society of America.

[32]  J. Stroop.  Studies of interference in serial verbal reactions. , 1992 .