Artificial Vision for the Blind: a Bio-Inspired Algorithm for Objects and Obstacles Detection

Although artificial vision systems could potentially provide very useful input to assistive devices for blind people, such devices are rarely used outside of laboratory experiments. Many current systems attempt to reproduce the visual image via an alternative sensory modality (often auditory or somatosensory), but this dominant "scoreboard" approach, is often difficult to interpret for the user. Here, we propose to offload the recognition problem onto a separate image processing system that then provides the user with just the essential information about the location of objects in the surrounding environment. Specifically, we show that a bio-inspired image processing algorithm (SpikeNet) can not only robustly, precisely, and rapidly recognize and locate key objects in the image, but also in space if the objects are in a stereoscopic field of view. In addition, the bio-inspired algorithm allows real-time calculation of optic flow. We hence propose that this system, coupled with a restitution interface allowing localization in space (i.e. three-dimensional virtual sounds synthesis) can be used to restore essential visuomotor behaviors such as grasping desired objects and navigating (finding directions, avoiding obstacles) in unknown environments.

[1]  S. Thorpe,et al.  Surfing a spike wave down the ventral stream , 2002, Vision Research.

[2]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[3]  M. Wanet-Defalque,et al.  Auditory coding of visual patterns for the blind. , 1999, Perception.

[4]  B. Katz,et al.  Auditory‐guided reaching movements in the peripersonal frontal space , 2008 .

[5]  Roberta L. Klatzky,et al.  Personal guidance system for the visually impaired , 1994, ASSETS.

[6]  Marie-Chantal Wanet-Defalque,et al.  Auditory Coding of Visual Patterns for the Blind , 1999 .

[7]  Frank Dellaert,et al.  SWAN: System for Wearable Audio Navigation , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[8]  Christophe Jouffrais,et al.  Assistive device for the blind based on object recognition: an application to identify currency bills , 2009, Assets '09.

[9]  Paul Bach-y-Rita,et al.  Seeing with the Brain , 2003, Int. J. Hum. Comput. Interact..

[10]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[11]  Rufin VanRullen,et al.  Temporal codes and sparse representations: A key to understanding rapid processing in the visual system , 2004, Journal of Physiology-Paris.

[12]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[13]  Denis G. Pelli,et al.  Visual requirements of mobility (A) , 1983 .

[14]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[15]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  Simon J. Thorpe,et al.  Ultra-Rapid Scene Categorization with a Wave of Spikes , 2002, Biologically Motivated Computer Vision.

[17]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[18]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[19]  R C Garry,et al.  The absorption of water from the colon of the rat under urethane anaesthesia , 1940, The Journal of physiology.

[20]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[21]  M Shinohara,et al.  Three-dimensional tactile display for the blind. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[22]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[23]  D. M. Green,et al.  Sound localization by human listeners. , 1991, Annual review of psychology.

[24]  J. O'Regan,et al.  Learning to Perceive with a Visuo — Auditory Substitution System: Localisation and Object Recognition with ‘The Voice’ , 2007, Perception.

[25]  M. Paré,et al.  Early-blind human subjects localize sound sources better than sighted subjects , 1998, Nature.

[26]  Roberta L. Klatzky,et al.  Evaluation of spatial displays for navigation without sight , 2006, TAP.