Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review

Given the growth in the numbers of visually impaired (VI) people in low-income countries, the development of affordable electronic travel aid (ETA) systems employing devices, sensors, and apps embedded in ordinary smartphones becomes a potentially cost-effective and reasonable all-in-one solution of utmost importance for the VI. This paper offers an overview of recent ETA research prototypes that employ smartphones for assisted orientation and navigation in indoor and outdoor spaces by providing additional information about the surrounding objects. Scientific achievements in the field were systematically reviewed using PRISMA methodology. Comparative meta-analysis showed how various smartphone-based ETA prototypes could assist with better orientation, navigation, and wayfinding in indoor and outdoor environments. The analysis found limited interest among researchers in combining haptic interfaces and computer vision capabilities in smartphone-based ETAs for the blind, few attempts to employ novel state-of-the-art computer vision methods based on deep neural networks, and no evaluations of existing off-the-shelf navigation solutions. These results were contrasted with findings from a survey of blind expert users on their problems in navigating in indoor and outdoor environments. This revealed a major mismatch between user needs and academic development in the field.

[1]  Lin Zhang,et al.  Integrated IMU with Faster R-CNN Aided Visual Measurements from IP Cameras for Indoor Positioning , 2018, Sensors.

[2]  Rabia Jafri,et al.  Computer vision-based object recognition for the visually impaired in an indoors environment: a survey , 2013, The Visual Computer.

[3]  James M. Coughlan,et al.  Towards a Real-Time System for Finding and Reading Signs for Visually Impaired Users , 2012, ICCHP.

[4]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[5]  Tony Stockman,et al.  A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research , 2015, Journal on Multimodal User Interfaces.

[6]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[7]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[8]  R. Klatzky,et al.  COGNITIVE MAPPING AND WAYFINDING BY ADULTS WITHOUT VISION , 1996 .

[9]  Maria Chiara Carrozza,et al.  Haptic-assistive technologies for audition and vision sensory disabilities , 2018, Disability and rehabilitation. Assistive technology.

[10]  James M. Coughlan,et al.  Crosswatch: A Camera Phone System for Orienting Visually Impaired Pedestrians at Traffic Intersections , 2008, ICCHP.

[11]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[12]  Andrius Budrionis,et al.  Indoor Navigation Systems for Visually Impaired Persons: Mapping the Features of Existing Technologies to User Needs , 2020, Sensors.

[13]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ruxandra Tapu,et al.  DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance , 2017, Sensors.

[15]  Sophie Hill,et al.  Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects , 2017, Research Involvement and Engagement.

[16]  Gretchen A. Stevens,et al.  Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. , 2017, The Lancet. Global health.

[17]  Ruxandra Tapu,et al.  When Ultrasonic Sensors and Computer Vision Join Forces for Efficient Obstacle Detection and Recognition , 2016, Sensors.

[18]  Alvaro Araujo,et al.  Navigation Systems for the Blind and Visually Impaired: Past Work, Challenges, and Open Problems , 2019, Sensors.

[19]  Roberto Manduchi,et al.  Mobile Vision as Assistive Technology for the Blind: An Experimental Study , 2012, ICCHP.

[20]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[21]  Giovanni Fusco,et al.  Determining a Blind Pedestrian's Location and Orientation at Traffic Intersections , 2014, ICCHP.

[22]  Phil Edwards,et al.  The knowledge system underpinning healthcare is not fit for purpose and must change , 2015, BMJ : British Medical Journal.

[23]  Rainer Stiefelhagen,et al.  An Assistive Vision System for the Blind That Helps Find Lost Things , 2012, ICCHP.

[24]  Andrew S. Pullin,et al.  The Policy Role of Systematic Reviews: Past, Present and Future , 2014, Springer Science Reviews.

[25]  Vytautas Daniulaitis,et al.  Segmentation of a Vibro-Shock Cantilever-Type Piezoelectric Energy Harvester Operating in Higher Transverse Vibration Modes , 2015, Sensors.

[26]  Narayanan Vijaykrishnan,et al.  A Multitask Grocery Assist System for the Visually Impaired: Smart glasses, gloves, and shopping carts provide auditory and tactile feedback , 2017, IEEE Consumer Electronics Magazine.

[27]  D G Altman,et al.  The scandal of poor medical research , 1994, BMJ.

[28]  Giovanni Fusco,et al.  Indoor Localization Using Computer Vision and Visual-Inertial Odometry , 2018, ICCHP.

[29]  SchmidhuberJürgen.  Deep learning in neural networks , 2015 .

[30]  Anthea M Burnett,et al.  Global Prevalence of Presbyopia and Vision Impairment from Uncorrected Presbyopia: Systematic Review, Meta-analysis, and Modelling. , 2018, Ophthalmology.

[31]  P. Shekelle,et al.  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement , 2015, Systematic Reviews.

[32]  R. Riera,et al.  Overview of systematic reviews - a new type of study. Part II. , 2014, Sao Paulo medical journal = Revista paulista de medicina.

[33]  Nicolette de Keizer,et al.  STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics , 2009, Yearbook of Medical Informatics.

[34]  Charles X. Ling,et al.  Pelee: A Real-Time Object Detection System on Mobile Devices , 2018, NeurIPS.

[35]  Michael Nunns,et al.  Abstracts from the NIHR INVOLVE Conference 2017 , 2017, Research Involvement and Engagement.

[36]  Manuela Chessa,et al.  An integrated artificial vision framework for assisting visually impaired users , 2016, Comput. Vis. Image Underst..

[37]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[38]  Khaled M. Elleithy,et al.  Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions , 2017, Sensors.

[39]  Tari Turner,et al.  Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap , 2014, PLoS medicine.

[40]  Mickael Bech,et al.  A MODEL FOR ASSESSMENT OF TELEMEDICINE APPLICATIONS: MAST , 2012, International Journal of Technology Assessment in Health Care.

[41]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[42]  György Wersényi,et al.  Overview of auditory representations in human-machine interfaces , 2013, ACM Comput. Surv..

[43]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.