PantoGuide: A Haptic and Audio Guidance System To Support Tactile Graphics Exploration

The ability to effectively read and interpret tactile graphics and charts is an essential part of a tactile learner’s path to literacy, but is a skill that requires instruction and training. Many teachers of the visual impaired (TVIs) report that blind and visually impaired students have trouble interpreting graphics independently without individual instruction. We present PantoGuide, a low-cost system that provides audio and haptic guidance, via skin-stretch feedback to the dorsum of a user’s hand while the user explores a tactile graphic overlaid on a touchscreen. This system allows programming of haptic guidance patterns and cues for tactile graphics that can be experienced by students learning remotely or that can be reviewed by a student independently. We propose two teaching scenarios (synchronous and asynchronous) and two guidance interactions (point-to-point and continuous) that the device can support – and demonstrate their use in a set of applications we co-design with one co-author who is blind and a tactile graphics user.

[1]  Kim T. Zebehazy,et al.  Quality, Importance, and Instruction: The Perspectives of Teachers of Students with Visual Impairments on Graphics Use by Students , 2014 .

[2]  Carole R. Beal,et al.  Teachers of Students with Visual Impairments Share Experiences and Advice for Supporting Students in Understanding Graphics , 2018, Journal of Visual Impairment & Blindness.

[3]  Stephen A. Brewster,et al.  Exploring computer-generated line graphs through virtual touch , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[4]  Mark D. Gross,et al.  FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile Markers , 2017, ASSETS.

[5]  Hideaki Kuzuoka,et al.  Development of Wearable Outer-Covering Haptic Display Using Ball Effector for Hand Motion Guidance , 2014, AsiaHaptics.

[6]  Richard E. Ladner,et al.  Tactile graphics with a voice: using QR codes to access text in tactile graphics , 2014, ASSETS.

[7]  William R. Provancher,et al.  Planar Hand Motion Guidance Using Fingertip Skin-Stretch Feedback , 2014, IEEE Transactions on Haptics.

[8]  Domenico Prattichizzo,et al.  A remote guidance system for blind and visually impaired people via vibrotactile haptic feedback , 2014, 22nd Mediterranean Conference on Control and Automation.

[9]  Carole R. Beal,et al.  Evaluation of the Effectiveness of a Tablet Computer Application (App) in Helping Students with Visual Impairments Solve Mathematics Problems , 2018 .

[10]  Sean Follmer,et al.  Editing Spatial Layouts through Tactile Templates for People with Visual Impairments , 2019, CHI.

[11]  Jane N. Erin,et al.  Using the Talking Tactile Tablet as a Testing Accommodation , 2007 .

[12]  Giovanni Fusco,et al.  The Tactile Graphics Helper: Providing Audio Clarification for Tactile Graphics Using Machine Vision , 2015, ASSETS.

[13]  Robert W. Lindeman,et al.  Haptic-assisted guidance system for navigating volumetric data sets , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[14]  Thorsten Schwarz,et al.  Understanding what you feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment , 2020, CHI.

[15]  Claudio Pacchierotti,et al.  Design and Evaluation of a Wearable Skin Stretch Device for Haptic Guidance , 2018, IEEE Robotics and Automation Letters.

[16]  James M. Coughlan,et al.  CamIO: a 3D computer vision system enabling audio/haptic interaction with physical objects by blind users , 2013, ASSETS.

[17]  Aaron M. Dollar,et al.  Outdoor pedestrian navigation assistance with a shape-changing haptic interface and comparison with a vibrotactile device , 2016, 2016 IEEE Haptics Symposium (HAPTICS).

[18]  Allison M. Okamura,et al.  Holdable Haptic Device for 4-DOF Motion Guidance , 2019, 2019 IEEE World Haptics Conference (WHC).

[19]  Vincent Hayward,et al.  International Conference on Intelligent Robots and Systems The Pantograph Mk-II : A Haptic Instrument * , 2005 .

[20]  Jun Rekimoto,et al.  Traxion: a tactile interaction device with virtual force sensation , 2013, SIGGRAPH '14.

[21]  Enrico Pontelli,et al.  Multimodal Presentation of Two-Dimensional Charts: An Investigation Using Open Office XML and Microsoft Excel , 2010, TACC.

[22]  G. Campion.  The Synthesis of Three Dimensional Haptic Textures: Geometry, Control, and Psychophysics , 2011 .

[23]  Marjorie S. Hooper.  American Printing House for the Blind , 1951 .

[24]  Kim T. Zebehazy,et al.  Charting Success: The Experience of Teachers of Students with Visual Impairments in Promoting Student Use of Graphics , 2014 .

[25]  L. Penny Rosenblum,et al.  Supporting Elementary School Students with Visual Impairments in Science Classes , 2019, Journal of Visual Impairment & Blindness.

[26]  Darren Guinness,et al.  Exploring the Design of Audio-Kinetic Graphics for Education , 2018, ICMI.

[27]  Valerie S. Morash.  The Relationship between Tactile Graphics and Mathematics for Students with Visual Impairments , 2014 .

[28]  Bing Wu,et al.  FingerSight: Fingertip Haptic Sensing of the Visual Environment , 2014, IEEE Journal of Translational Engineering in Health and Medicine.

[29]  Stephen A. Brewster,et al.  Comparing two haptic interfaces for multimodal graph rendering , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[30]  Robert Wall Emerson,et al.  Teachers’ Descriptions of Mathematics Graphics for Students with Visual Impairments: A Preliminary Investigation , 2020 .