ActiVis: Mobile Object Detection and Active Guidance for People with Visual Impairments

The ActiVis project aims to deliver a mobile system that is able to guide a person with visual impairments towards a target object or area in an unknown indoor environment. For this, it uses new developments in object detection, mobile computing, action generation and human-computer interfacing to interpret the user’s surroundings and present effective guidance directions. Our approach to direction generation uses a Partially Observable Markov Decision Process (POMDP) to track the system’s state and output the optimal location to be investigated. This system includes an object detector and an audio-based guidance interface to provide a complete active search pipeline. The ActiVis system was evaluated in a set of experiments showing better performance than a simpler unguided case.

[1]  Jeffrey P. Bigham,et al.  VizWiz::LocateIt - enabling blind people to locate objects in their environment , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[2]  Rainer Stiefelhagen,et al.  An Assistive Vision System for the Blind That Helps Find Lost Things , 2012, ICCHP.

[3]  R. Butler,et al.  Factors that influence the localization of sound in the vertical plane. , 1968, The Journal of the Acoustical Society of America.

[4]  Paolo Gallina,et al.  Progressive co-adaptation in human-machine interaction , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[5]  John K. Tsotsos,et al.  Revisiting active perception , 2016, Autonomous Robots.

[6]  Grzegorz Cielniak,et al.  Portable navigations system with adaptive multimodal interface for the blind , 2017, AAAI 2017.

[7]  Weiyao Lin,et al.  Tiny-DSOD: Lightweight Object Detection for Resource-Restricted Usages , 2018, BMVC.

[8]  Tadej Petric,et al.  Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation , 2018, Auton. Robots.

[9]  Jordi Pont-Tuset,et al.  The Open Images Dataset V4 , 2018, International Journal of Computer Vision.

[10]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[11]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[12]  Nicola Bellotto.  A multimodal smartphone interface for active perception by visually impaired , 2013 .

[13]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[15]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[16]  Gretchen A. Stevens,et al.  Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. , 2017, The Lancet. Global health.

[17]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Roberto Manduchi,et al.  Mobile Vision as Assistive Technology for the Blind: An Experimental Study , 2012, ICCHP.

[19]  Jana Kosecka,et al.  A dataset for developing and benchmarking active vision , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[22]  Grzegorz Cielniak,et al.  Active Object Search with a Mobile Device for People with Visual Impairments , 2019, VISIGRAPP.