Looktel—A comprehensive platform for computer-aided visual assistance

We present an extensible platform that integrates state of the art computer vision techniques with mobile communications to deliver a portable visual assistance tool. Live input video from a mobile smartphone is streamed over a 3G or wireless connection while an object recognition engine on a desktop processes the data stream. Recognition results are returned in real-time to the mobile device and announced by a text-to-speech engine. The system design is complete and includes the ability to add new items, share databases, and provide live remote human sighted assistance.

[1]  Jana Kosecka,et al.  Landmark-Based Pedestrian Navigation with Enhanced Spatial Reasoning , 2009, Pervasive.

[2]  Jana Kosecka,et al.  Global localization and relative positioning based on scale-invariant keypoints , 2005, Robotics Auton. Syst..

[3]  Alan L. Yuille,et al.  AdaBoost Learning for Detecting and Reading Text in City Scenes , 2004, CVPR 2004.

[4]  Bernd Girod,et al.  Streaming mobile augmented reality on mobile phones , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[5]  Natasha Gelfand,et al.  SURFTrac: Efficient tracking and continuous object recognition using local feature descriptors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Paolo Pirjanian,et al.  SIFT-ing through features with ViPR , 2006, IEEE Robotics & Automation Magazine.

[7]  Jonathan Allen,et al.  Text to Speech , 2015 .

[8]  Bernd Girod,et al.  Low-rate image retrieval with tree histogram coding , 2009, MobiMedia.

[9]  Roberto Manduchi,et al.  Functional Assessment of a Camera Phone-Based Wayfinding System Operated by Blind and Visually Impaired Users , 2009, Int. J. Artif. Intell. Tools.

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  Allen R. Hanson,et al.  Automatic Sign Detection and Recognition in Natural Scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[12]  Allen R. Hanson,et al.  Sign Classification using Local and Meta-Features , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.