Toward haptic/aural touchscreen display of graphical mathematics for the education of blind students

We propose the use of a haptic touchscreen to convey graphical and mathematical concepts through aural and/or vibratory tactile feedback. We hypothesize that an important application of such a display will be in teaching visually impaired students concepts that are traditionally learned almost entirely visually. This paper describes initial feasibility studies using a commercially available haptic touchscreen to display grids, points, lines, and shapes - some of the first visual graphical entities students encounter in K-12 mathematics education, and from which more complex lessons can be constructed. We conducted user studies designed to evaluate perception of these objects through haptic feedback alone, auditory feedback alone, and combinations of the two. Our results indicate that both sensory channels can be valuable in user perception.

[1]  R. Klatzky,et al.  Visual mediation and the haptic recognition of two-dimensional pictures of common objects , 1990, Perception & psychophysics.

[2]  S. Ungar,et al.  The Construction of Cognitive Maps by Children with Visual Impairments , 1996 .

[3]  Stephen A. Brewster,et al.  Constructing sonified haptic line graphs for the blind student: first steps , 2000, Assets '00.

[4]  Christopher R. Wagner,et al.  A tactile shape display using RC servomotors , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[5]  Hesham M. Kamel,et al.  A Comparison of Three Nonvisual Methods for Presenting Scientific Graphs , 2002 .

[6]  Vincent Hayward,et al.  STReSS: A Practical Tactile Display System with One Millimeter Spatial Resolution and 700 Hz Refresh Rate , 2003 .

[7]  Stephen A. Brewster,et al.  Evaluation of multimodal graphs for blind people , 2003, Universal Access in the Information Society.

[8]  Stephen A. Brewster,et al.  Web-based haptic applications for blind people to create virtual graphs , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[9]  Ivan Poupyrev,et al.  Tactile interfaces for small touch screens , 2003, UIST '03.

[10]  Roope Raisamo,et al.  Perception strategies in modal-redistributed interaction , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[11]  Roope Raisamo,et al.  Non-visual feedback cues for pen computing , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[12]  S. Ono,et al.  Practical use of interactive tactile graphic display system at a school for the blind (2) , 2006 .

[13]  Michael Wiertlewski,et al.  A compact tactile display for the blind with shape memory alloys , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[14]  Lisa Gjedde,et al.  Current developments in technology-assisted education , 2006 .

[15]  Charlotte Magnusson,et al.  User Evaluations of a Virtual Haptic-Audio Line Drawing Prototype , 2006, HAID.

[16]  Gi-Hun Yang,et al.  Quantitative tactile display device with pin-array type tactile feedback and thermal feedback , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[17]  Dong-Soo Kwon,et al.  Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator , 2007, Adv. Robotics.

[18]  F. Vidal-Verdu,et al.  Graphical Tactile Displays for Visually-Impaired People , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  J. Edward Colgate,et al.  T-PaD: Tactile Pattern Display through Variable Friction Reduction , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[20]  Stephen A. Brewster,et al.  Investigating touchscreen accessibility for people with visual impairments , 2008, NordiCHI.

[21]  Stephen A. Brewster,et al.  Investigating the effectiveness of tactile feedback for mobile touchscreens , 2008, CHI.

[22]  Stephen A. Brewster,et al.  Multimodal Trajectory Playback for Teaching Shape Information and Trajectories to Visually Impaired Computer Users , 2008, TACC.

[23]  Gerhard Weber,et al.  Tactile Graphics Revised: The Novel BrailleDis 9000 Pin-Matrix Device with Multitouch Input , 2008, ICCHP.

[24]  David Mioduser,et al.  Haptic-feedback support for cognitive mapping of unknown spaces by people who are blind , 2008, Int. J. Hum. Comput. Stud..

[25]  Tae-Heon Yang,et al.  Development of a miniature pin-array tactile module using elastic and electromagnetic force for mobile devices , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[26]  Sebastian Merchel,et al.  Audiotactile Feedback Design for Touch Screens , 2009, HAID.

[27]  Michael Levin,et al.  Tactile-Feedback Solutions for an Enhanced User Experience , 2009 .

[28]  J. Edward Colgate,et al.  ShiverPaD: A Glass Haptic Surface That Produces Shear Force on a Bare Finger , 2010, IEEE Transactions on Haptics.