Assisting the visually impaired using depth inference on mobile devices via stereo matching

A novel system for assisting the visually impaired, built upon the platform of a mobile device, is proposed. The system uses depth inference and a depth-to-sound mapping to inform the user of the surroundings via sound. Stereo matching is proposed as the appropriate tool for depth inference, and stereo matching approaches are assessed for their potential for real-time performance on a mobile device and their inference quality. A novel depth-to-sound mapping is proposed that can be adjusted by the user through the touch interface of the mobile device. This mapping allows the user to define the field of view and the location of inference. A smart phone “app” and demo will be produced. The system demonstrates the viability and great benefit of depth inference on mobile devices for assisting the visually impaired.

[1]  Yi Deng,et al.  A Fast Line Segment Based Dense Stereo Algorithm Using Tree Dynamic Programming , 2006, ECCV.

[2]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[3]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Roberta L. Klatzky,et al.  Stated Preferences for Components of a Personal Guidance System for Nonvisual Navigation , 2004 .

[5]  Myounghoon Jeon,et al.  "Listen2dRoom": helping blind individuals understand room layouts , 2012, CHI EA '12.

[6]  Minh N. Do,et al.  A revisit to cost aggregation in stereo matching: How far can we reduce its computational redundancy? , 2011, 2011 International Conference on Computer Vision.

[7]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Kari Pulli,et al.  Real-time computer vision with OpenCV , 2012, Commun. ACM.

[9]  Gert Jan Gelderblom,et al.  Inventory of Electronic Mobility Aids for Persons with Visual Impairments: A Literature Review , 2008 .

[10]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  John J. Rieser,et al.  Blindness and Brain Plasticity in Navigation and Object Perception , 2007 .

[12]  Gérard G. Medioni,et al.  Robot vision for the visually impaired , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.