Echolocation as a Means for People with Visual Impairment (PVI) to Acquire Spatial Knowledge of Virtual Space

In virtual environments, spatial information is communicated visually. This prevents people with visual impairment (PVI) from accessing such spaces. In this article, we investigate whether echolocation could be used as a tool to convey spatial information by answering the following research questions: What features of virtual space can be perceived by PVI through the use of echolocation? How does active echolocation support PVI in acquiring spatial knowledge of a virtual space? And what are PVI’s opinions regarding the use of echolocation to acquire landmark and survey knowledge of virtual space? To answer these questions, we conducted a two-part within-subjects experiment with 12 people who were blind or had a visual impairment and found that size and materials of rooms and 90-degree turns were detectable through echolocation, participants preferred using echoes derived from footsteps rather than from artificial sound pulses, and echolocation supported the acquisition of mental maps of a virtual space. Ultimately, we propose that appropriately designed echolocation in virtual environments improves understanding of spatial information and access to digital games for PVI.

[1]  Luciana Porcher Nedel,et al.  Inclusive Games: A Multimodal Experience for Blind Players , 2011, 2011 Brazilian Symposium on Games and Digital Entertainment.

[2]  R. Klatzky,et al.  COGNITIVE MAPPING AND WAYFINDING BY ADULTS WITHOUT VISION , 1996 .

[3]  Christ Glorieux,et al.  Localization of a virtual wall by means of active echolocation by untrained sighted persons , 2018, Applied Acoustics.

[4]  Lawrence D. Rosenblum,et al.  Echolocating Distance by Moving and Stationary Listeners , 2000 .

[5]  L. Thaler,et al.  Navigation and perception of spatial layout in virtual echo-acoustic space , 2020, Cognition.

[6]  Carl Gutwin,et al.  Threats of a replication crisis in empirical computer science , 2020, Commun. ACM.

[7]  Orly Lahav,et al.  A Virtual Environment for People Who Are Blind - A Usability Study. , 2012, Journal of assistive technologies.

[8]  Lore Thaler,et al.  Echolocation in humans: an overview. , 2016, Wiley interdisciplinary reviews. Cognitive science.

[9]  Jennifer L. Milne,et al.  Shape-specific activation of occipital cortex in an early blind echolocation expert , 2013, Neuropsychologia.

[10]  S H FEINSTEIN,et al.  ECHO-DETECTION ABILITY OF THE BLIND: SIZE AND DISTANCE FACTORS. , 1965, Journal of experimental psychology.

[11]  Nashwa El-Bendary,et al.  Virtual Reality Technology for Blind and Visual Impaired People: Reviews and Recent Advances , 2012 .

[12]  Tyler Thrash,et al.  Spatial navigation by congenitally blind individuals , 2015, Wiley interdisciplinary reviews. Cognitive science.

[13]  Christopher Frauenberger,et al.  Disability and Technology: A Critical Realist Perspective , 2015, ASSETS.

[14]  Graham R. Gibbs,et al.  Thematic Coding and Categorizing , 2007 .

[15]  Sabine Timpf,et al.  The Landmark Spider: Representing Landmark Knowledge for Wayfinding Tasks , 2005, AAAI Spring Symposium: Reasoning with Mental and External Diagrams: Computational Modeling and Spatial Assistance.

[16]  Pablo Luis López Espí,et al.  Physical Analysis of Several Organic Signals for Human Echolocation: Oral Vacuum Pulses , 2009 .

[17]  Jenny Waycott,et al.  Echo-house: exploring a virtual environment by using echolocation , 2018, OZCHI.

[18]  Matt Wilkerson,et al.  Does a sonar system make a blind maze navigation computer game more "fun"? , 2010, ASSETS '10.

[19]  Claudia Arias,et al.  Human echolocation: The ECOTEST system , 1997 .

[20]  Ravish Mehra,et al.  Comparison of localization performance with individualized and non-individualized head-related transfer functions for dynamic listeners , 2016 .

[21]  Michel Denis,et al.  Referring to Landmark or Street Information in Route Directions: What Difference Does It Make? , 2003, COSIT.

[22]  David Whitney,et al.  Ultrafine spatial acuity of blind expert human echolocators , 2012, Experimental Brain Research.

[23]  Luigi F. Cuturi,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[24]  J. B. Pittenger,et al.  Human Echolocation as a Basic Form of Perception and Action , 1995 .

[25]  Florian Güldenpfennig,et al.  Unpacking the Audio Game Experience: Lessons Learned from Game Veterans , 2019, CHI PLAY.

[26]  Daniel Rowan,et al.  Identification of the lateral position of a virtual object based on echoes by humans , 2013, Hearing Research.

[27]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[28]  B. Belland Scaffolding: Definition, Current Debates, and Future Directions , 2014 .

[29]  Gianni Virgili,et al.  Orientation and mobility training for adults with low vision. , 2010, The Cochrane database of systematic reviews.

[30]  Lore Thaler,et al.  A blind human expert echolocator shows size constancy for objects perceived by echoes , 2015, Neurocase.

[31]  Lore Thaler,et al.  Echolocation may have real-life advantages for blind people: an analysis of survey data , 2013, Front. Physiol..

[32]  Raja Syamsul Azmir Raja Abdullah,et al.  Mouth-clicks used by blind expert human echolocators – signal description and model based signal synthesis , 2017, PLoS Comput. Biol..

[33]  T. Shakespeare,et al.  Disability Rights and Wrongs , 2009 .

[34]  Annika Neidhardt Detection of a Nearby Wall in a Virtual Echolocation Scenario Based on Measured and Simulated OBRIRS , 2018 .

[35]  L. Wiegrebe,et al.  Human Exploration of Enclosed Spaces through Echolocation , 2017, The Journal of Neuroscience.

[36]  David Mioduser,et al.  Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blind , 2008, Comput. Hum. Behav..

[37]  Albert Postma,et al.  Categorical and metric distance information in mental representations derived from route and survey descriptions , 2005, Psychological research.

[38]  John W. Creswell,et al.  Designing and Conducting Mixed Methods Research , 2006 .

[39]  Ming-Puu Chen,et al.  The effects of goal specificity and scaffolding on programming performance and self-regulation in game design , 2014, Br. J. Educ. Technol..

[40]  T. Shakespeare Disability Rights and Wrongs Revisited , 2013 .

[41]  Alexis Kirke,et al.  When the Soundtrack Is the Game: From Audio-Games to Gaming the Music , 2018 .

[42]  Karin Schweizer,et al.  Spatial Cognition: The Role of Landmark, Route, and Survey Knowledge in Human and Robot Navigation , 1997, GI Jahrestagung.

[43]  Vicki L. Hanson,et al.  PowerUp: an accessible virtual world , 2008, Assets '08.

[44]  Cumhur Erkut,et al.  The Effects of Ecological Auditory Feedback on Rhythmic Walking Interaction , 2015, IEEE MultiMedia.

[45]  Virginia Braun,et al.  Successful Qualitative Research: A practical guide for beginners , 2013, QMiP Bulletin.

[46]  Brian C. J. Moore,et al.  A summary of research investigating echolocation abilities of blind and sighted humans , 2014, Hearing Research.

[47]  Florian von Zabiensky,et al.  Ultrasonic Waves to Support Human Echolocation , 2018, HCI.

[48]  Bei Yuan,et al.  Game accessibility: a survey , 2011, Universal Access in the Information Society.

[49]  Luca Turchet,et al.  Sound Synthesis and Evaluation of Interactive Footsteps and Environmental Sounds Rendering for Virtual Reality Applications , 2011, IEEE Transactions on Visualization and Computer Graphics.

[50]  Anke M. Brock,et al.  Interactivity Improves Usability of Geographic Maps for Visually Impaired People , 2015, Hum. Comput. Interact..

[51]  Anke M. Brock,et al.  Towards a Multisensory Augmented Reality Map for Blind and Low Vision People: a Participatory Design Approach , 2018, CHI.

[52]  Ewout W Steyerberg,et al.  Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes , 2011, BMC medical research methodology.

[53]  Helene Joffe,et al.  Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines , 2020 .

[54]  Lutz Wiegrebe,et al.  Self-motion facilitates echo-acoustic orientation in humans , 2014, Royal Society Open Science.

[55]  João Guerreiro,et al.  Virtual Navigation for Blind People: Building Sequential Representations of the Real-World , 2017, ASSETS.

[56]  Leona Holloway,et al.  Accessible Maps for the Blind: Comparing 3D Printed Models with Tactile Graphics , 2018, CHI.

[57]  Julie A. Kientz,et al.  An empirical study of issues and barriers to mainstream video game accessibility , 2013, ASSETS.

[58]  K. M. Dallenbach,et al.  "Facial Vision": The Perception of Obstacles by the Blind , 1944 .

[59]  Manohar Swaminathan,et al.  Video Gaming for the Vision Impaired , 2018, ASSETS.

[60]  Helmut Hlavacs,et al.  Blind Adventure - A Game Engine for Blind Game Designers , 2018, CHI PLAY.

[61]  Cumhur Erkut,et al.  An investigation on the impact of auditory and haptic feedback on rhythmic walking interactions , 2016, Int. J. Hum. Comput. Stud..

[62]  Mike Wald,et al.  Map data representation for indoor navigation by blind people , 2016 .

[63]  Lynette van Zijl,et al.  Virtual World Accessibility with the Perspective Viewer , 2015 .

[64]  Mihoko Niitsuma,et al.  Mental map generation assistance tool using relative pitch difference and angular information for visually impaired people , 2013, 2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom).

[65]  Brian F. G. Katz,et al.  Impact of HRTF Individualization on Player Performance in a VR Shooter Game I , 2018 .

[66]  Jenny Waycott,et al.  Playing Blind: Revealing the World of Gamers with Visual Impairment , 2019, CHI.

[67]  Chuen-Tsai Sun,et al.  Associations among scaffold presentation, reward mechanisms and problem-solving behaviors in game play , 2018, Comput. Educ..

[68]  Barry Peterson,et al.  Spatial Orientation, Wayfinding, and Representation , 2014, Handbook of Virtual Environments, 2nd ed..

[69]  R. Mclean,et al.  A Unified Approach to Mixed Linear Models , 1991 .

[70]  Bei Yuan,et al.  Blind hero: enabling guitar hero for the visually impaired , 2008, Assets '08.

[71]  Sanford Weisberg,et al.  An R Companion to Applied Regression , 2010 .

[72]  Jaime Sánchez,et al.  Action video game play and transfer of navigation and spatial cognition skills in adolescents who are blind , 2014, Front. Hum. Neurosci..

[73]  W. Jacobson The Art and Science of Teaching Orientation and Mobility to Persons With Visual Impairments , 1993 .

[74]  Jeff Sauro,et al.  Quantifying the User Experience: Practical Statistics for User Research , 2012 .

[75]  W. N. Kellogg,et al.  Sonar system of the blind. , 1962, Science.

[76]  C. Thinus-Blanc,et al.  Representation of space in blind persons: vision as a spatial sense? , 1997, Psychological bulletin.

[77]  Mary C. Whitton,et al.  Walking > walking-in-place > flying, in virtual environments , 1999, SIGGRAPH.

[78]  Bruno Sinopoli,et al.  EchoExplorer(TM): A Game App for Understanding Echolocation and Learning to Navigate Using Echo Cues , 2017 .

[79]  M. Goodale,et al.  Citation for Published Item: Use Policy Neural Correlates of Natural Human Echolocation in Early and Late Blind Echolocation Experts , 2022 .

[80]  James L. Alty,et al.  Communicating graphical information to blind users using music: the role of context , 1998, CHI.

[81]  K. M. Dallenbach,et al.  "Facial vision:" the role of pitch and loudness in the perception of obstacles by the blind. , 1950, The American journal of psychology.

[82]  Michel Denis,et al.  Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge , 2014, Int. J. Hum. Comput. Stud..

[83]  Kyle Rector,et al.  Virtual Showdown: An Accessible Virtual Reality Game with Scaffolds for Youth with Visual Impairments , 2019, CHI.

[84]  A. Siegel,et al.  The development of spatial representations of large-scale environments. , 1975, Advances in child development and behavior.

[85]  R. Kitchin,et al.  Techniques to Collect and Analyze the Cognitive Map Knowledge of Persons with Visual Impairment Or Blindness: Issues of Validity , 1997 .

[86]  Shree K. Nayar,et al.  The RAD: Making Racing Games Equivalently Accessible to People Who Are Blind , 2018, CHI.

[87]  Rudy Darken,et al.  Wayfinding strategies and behaviors in large virtual worlds , 1996, CHI.

[88]  M. Ptito,et al.  Compensatory plasticity and cross-modal reorganization following early visual deprivation , 2014, Neuroscience & Biobehavioral Reviews.

[89]  David Guth,et al.  Echolocation Reconsidered: Using Spatial Variations in the Ambient Sound Field to Guide Locomotion , 1998 .

[90]  Ana Tajadura-Jiménez,et al.  Generic HRTFs May be Good Enough in Virtual Reality. Improving Source Localization through Cross-Modal Plasticity , 2018, Front. Neurosci..

[91]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[92]  Jeremy N. Bailenson,et al.  The Unbearable Likeness of Being Digital: The Persistence of Nonverbal Social Norms in Online Virtual Environments , 2007, Cyberpsychology Behav. Soc. Netw..

[93]  Jenny Waycott,et al.  Introducing the Gamer Information-Control Framework: Enabling Access to Digital Games for People with Visual Impairment , 2020, CHI.

[94]  J. Huizinga,et al.  Homo Ludens Ils 86 , 2014 .