Toucha11y: Making Inaccessible Public Touchscreens Accessible

Despite their growing popularity, many public kiosks with touchscreens are inaccessible to blind people. Toucha11y is a working prototype that allows blind users to use existing inaccessible touchscreen kiosks independently and with little effort. Toucha11y consists of a mechanical bot that can be instrumented to an arbitrary touchscreen kiosk by a blind user and a companion app on their smartphone. The bot, once attached to a touchscreen, will recognize its content, retrieve the corresponding information from a database, and render it on the user’s smartphone. As a result, a blind person can use the smartphone’s built-in accessibility features to access content and make selections. The mechanical bot will detect and activate the corresponding touchscreen interface. We present the system design of Toucha11y along with a series of technical evaluations. Through a user study, we found out that Toucha11y could help blind users operate inaccessible touchscreen devices.

[1]  Huaishu Peng,et al.  TangibleGrid: Tangible Web Layout Design for Blind Users , 2022, UIST.

[2]  Vinita Tibdewal,et al.  ”Would the smart cane benefit me?”: Perceptions of the Visually Impaired towards Smart Canes , 2021, ASSETS.

[3]  E. Whiting,et al.  TangibleCircuits: An Interactive 3D Printed Circuit Education Tool for People with Visual Impairments , 2020, CHI.

[4]  D. Gurari,et al.  Captioning Images Taken by People Who Are Blind , 2020, ECCV.

[5]  Aaron Johnson,et al.  Exploring the use of smartphones and tablets among people with visual impairments: Are mainstream devices replacing the use of traditional visual aids? , 2019, Assistive technology : the official journal of RESNA.

[6]  Ellen Yi-Luen Do,et al.  ShapeBots: Shape-changing Swarm Robots , 2019, UIST.

[7]  Jeffrey P. Bigham,et al.  StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible , 2019, UIST.

[8]  Jonathan Lazar,et al.  Toward unified guidelines for kiosk accessibility , 2019, Interactions.

[9]  Darren Guinness,et al.  StoryBlocks: A Tangible Programming Game To Create Accessible Audio Stories , 2019, CHI.

[10]  Luis Guerrero,et al.  Adobe , 2018, The Encyclopedia of Archaeological Sciences.

[11]  Markus Funk,et al.  DroneNavigator: Using Leashed and Free-Floating Quadcopters to Navigate Visually Impaired Travelers , 2017, ASSETS.

[12]  Xiang 'Anthony' Chen,et al.  Facade: Auto-generating Tactile Interfaces to Appliances , 2017, CHI.

[13]  Khaled M. Elleithy,et al.  Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions , 2017, Sensors.

[14]  Ramiro Velazquez,et al.  Wearable Assistive Devices for the Blind , 2016, ArXiv.

[15]  Xiang 'Anthony' Chen,et al.  Facade: Auto-generating Tactile Interfaces to Appliances , 2016, ASSETS.

[16]  Majed Al Zayer,et al.  Exploring the Use of a Drone to Guide Blind Runners , 2016, ASSETS.

[17]  Jon Froehlich,et al.  Evaluating Haptic and Auditory Directional Guidance to Assist Blind People in Reading Printed Text Using Finger-Mounted Cameras , 2016, ACM Trans. Access. Comput..

[18]  Chieko Asakawa,et al.  VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World , 2016, UIST.

[19]  Yuhang Zhao,et al.  ForeSee: A Customizable Head-Mounted Vision Enhancement System for People with Low Vision , 2015, ASSETS.

[20]  Uran Oh,et al.  Current and future mobile and wearable device use by people with visual impairments , 2014, CHI.

[21]  Meredith Ringel Morris,et al.  Touchplates: low-cost tactile overlays for visually impaired touch screen users , 2013, ASSETS.

[22]  Jacob O. Wobbrock,et al.  Access lens: a gesture-based screen reader for real-world documents , 2013, CHI.

[23]  Suranga Nanayakkara,et al.  EyeRing: a finger-worn input device for seamless interactions with our surroundings , 2013, AH.

[24]  Hod Lipson,et al.  A Positive Pressure Universal Gripper Based on the Jamming of Granular Material , 2012 .

[25]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[26]  Ariffin Abdul Mutalib,et al.  Smart Cane: Assistive Cane for Visually-impaired People , 2011, ArXiv.

[27]  I. Scott MacKenzie,et al.  Eyes-free text entry with error correction on touchscreen mobile devices , 2010, NordiCHI.

[28]  Rob Miller,et al.  VizWiz: nearly real-time answers to visual questions , 2010, UIST.

[29]  Richard E. Ladner,et al.  Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities , 2009, Assets '09.

[30]  John Nicholson,et al.  ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode Scans , 2009 .

[31]  Jacob O. Wobbrock,et al.  Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques , 2008, Assets '08.

[32]  Pierre Dragicevic,et al.  Earpod: eyes-free menu selection using touch input and reactive audio feedback , 2007, CHI.

[33]  M. Ross,et al.  Clearspeech: A Display Reader for the Visually Handicapped , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  John Nicholson,et al.  Robot-assisted wayfinding for the visually impaired in structured indoor environments , 2006, Auton. Robots.

[35]  Grigori E. Evreinov,et al.  Adaptive blind interaction technique for touchscreens , 2006, Universal Access in the Information Society.

[36]  John Nicholson,et al.  RoboCart: toward robot-assisted navigation of grocery stores by the visually impaired , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[38]  John Nicholson,et al.  RFID in robot-assisted indoor navigation for the visually impaired , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[39]  Gregg C. Vanderheiden,et al.  Use of Audio-Haptic Interface Techniques to Allow Nonvisual Access to Touchscreen Appliances , 1996 .

[40]  Vinita Tibdewal,et al.  An Exploratory Study on the Low Adoption Rate of Smart Canes , 2022, Interacción.

[41]  Nikolaos G. Bourbakis,et al.  Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[42]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[43]  Christopher Hunt.  SURF: Speeded-Up Robust Features , 2009 .

[44]  Gregg C. Vanderheiden,et al.  Web Content Accessibility Guidelines (WCAG) 2.0 , 2008 .