Tactile Board: A Multimodal Augmentative and Alternative Communication Device for Individuals with Deafblindness

Deafblindness, also known as dual sensory loss, is the combination of sight and hearing impairments of such extent that it becomes difficult for one sense to compensate for the other. Communication issues are a key concern for the Deafblind community. We present the design and technical implementation of the Tactile Board: a mobile Augmentative and Alternative Communication (AAC) device for individuals with deafblindness. The Tactile Board allows text and speech to be translated into vibrotactile signs that are displayed real-time to the user via a haptic wearable. Our aim is to facilitate communication for the deafblind community, creating opportunities for these individuals to initiate and engage in social interactions with other people without the direct need of an intervener.

[1]  Hiroshi Ishii,et al.  Social Textiles: Social Affordances and Icebreaking Interactions Through Wearable Social Messaging , 2015, TEI.

[2]  Kaisa Väänänen,et al.  Expectations for user experience in haptic communication with mobile devices , 2009, Mobile HCI.

[3]  Janice Light,et al.  Communicative Competence for Individuals who require Augmentative and Alternative Communication: A New Definition for a New Era of Communication? , 2014, Augmentative and alternative communication.

[4]  Andreas Butz,et al.  Tactile feedback without a big fuss: simple actuators for high-resolution phantom sensations , 2011, UbiComp '11.

[5]  Marion A. Hersh,et al.  SmartFingerBraille: A tactile sensing and actuation based communication glove for deafblind people , 2017, 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE).

[6]  Arthur Theil,et al.  Keep Your Distance: A Playful Haptic Navigation Wearable for Individuals with Deafblindness , 2020, ASSETS.

[7]  Peter B. Shull,et al.  Haptic wearables as sensory replacement, sensory augmentation and trainer – a review , 2015, Journal of NeuroEngineering and Rehabilitation.

[8]  Ben J. A. Kröse,et al.  The TaSSt: Tactile sleeve for social touch , 2013, 2013 World Haptics Conference (WHC).

[9]  Gianpaolo Francesco Trotta,et al.  A Wearable Device Supporting Multiple Touch- and Gesture-Based Languages for the Deaf-Blind , 2017 .

[10]  I. Sakuma,et al.  Finger Braille recognition system for people who communicate with deafblind people , 2008, 2008 IEEE International Conference on Mechatronics and Automation.

[11]  Karon E. MacLean,et al.  Detecting vibrations across the body in mobile contexts , 2011, CHI.

[12]  Hiroshi Kato,et al.  Basic Properties of Phantom Sensation for Practical Haptic Applications , 2010, EuroHaptics.

[13]  Oliver Korn,et al.  Empowering Persons with Deafblindness: Designing an Intelligent Assistive Wearable in the SUITCEYES Project , 2018, PETRA.

[14]  Ravinder Dahiya,et al.  Wearable Assistive Tactile Communication Interface Based on Integrated Touch Sensors and Actuators , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Myrthe A. Plaisier,et al.  Learning the Vibrotactile Morse Code Alphabet , 2020, ACM Trans. Appl. Percept..

[16]  Ola Hendar,et al.  A CASE STUDY OF TACTILE LANGUAGE AND ITS POSSIBLE STRUCTURE: A Tentative Outline to Study Tactile Language Systems among Children with Congenital Deafblindness , 2015 .

[17]  Nobuyuki Sasaki,et al.  Helen Keller Phone—a communication system for deaf-blind people using Body-Braille and Skype , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[18]  Marion Hersh,et al.  Deafblind people, stigma and the use of communication and mobility assistive devices , 2013 .

[19]  Oscar Tomico,et al.  Tactile Dialogues: Personalization of Vibrotactile Behavior to Trigger Interpersonal Communication , 2015, TEI.

[20]  Arthur Theil,et al.  Co-Designing Assistive Tools to Support Social Interactions by Individuals Living with Deafblindness , 2020, Conference on Designing Interactive Systems.

[21]  Nobuyuki Sasaki,et al.  Communication System between Deaf-Blind People and Non-Disabled People Using Body-Braille and Infrared Communication , 2010, 2010 7th IEEE Consumer Communications and Networking Conference.

[22]  Lynette A. Jones,et al.  Development of a tactile vest , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[23]  Meredith Ringel Morris,et al.  Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective , 2019, ASSETS.

[24]  Michael T Clarke,et al.  Language learning, recasts, and interaction involving AAC: background and potential for intervention , 2017, Augmentative and alternative communication.

[25]  Joel C. Huegel,et al.  MyVox—Device for the communication between people: blind, deaf, deaf-blind and unimpaired , 2014, IEEE Global Humanitarian Technology Conference (GHTC 2014).

[26]  Gregory D. Abowd,et al.  An evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired , 2012, Mobile HCI.

[27]  Ali Israr,et al.  Tactile brush: drawing on skin with a tactile grid display , 2011, CHI.

[28]  Anne Marie Piper,et al.  Designing Wearable Haptic Information Displays for People with Vision Impairments , 2015, Tangible and Embedded Interaction.

[29]  Janice Light,et al.  Challenges and opportunities in augmentative and alternative communication: Research and technology development to enhance communication and participation for individuals with complex communication needs , 2019, Augmentative and alternative communication.

[30]  Tracy Anne Hammond,et al.  Designing Vibrotactile Codes to Communicate Verb Phrases , 2014, TOMM.

[31]  Eva Lindell,et al.  Physical Add-Ons for Haptic Human-Surrounding Interaction and Sensorial Augmentation , 2020 .

[32]  Seung-Chan Kim,et al.  Surround haptics: tactile feedback for immersive gaming experiences , 2012, CHI Extended Abstracts.

[33]  Oliver Korn,et al.  Tactile navigation with checkpoints as progress indicators?: only when walking longer straight paths , 2020, PETRA.

[34]  J C Craig,et al.  Vibrotactile pattern recognition and discrimination at several body sites , 1984, Perception & psychophysics.

[35]  Marion Hersh,et al.  Deafblind people, communication, independence, and isolation. , 2013, Journal of deaf studies and deaf education.

[36]  Maria Chiara Carrozza,et al.  Haptic-assistive technologies for audition and vision sensory disabilities , 2018, Disability and rehabilitation. Assistive technology.

[37]  Frank Vetere,et al.  Body as a Canvas: An Exploration on the Role of the Body as Display of Digital Information , 2018, Conference on Designing Interactive Systems.

[38]  Karon E. MacLean,et al.  Haptic phonemes: basic building blocks of haptic communication , 2006, ICMI '06.

[39]  Richard E. Ladner,et al.  V-braille: haptic braille perception using a touch-screen and vibration on mobile phones , 2010, ASSETS '10.

[40]  Erin Piateski,et al.  Vibrotactile pattern recognition on the arm and torso , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[41]  Arthur Theil,et al.  Exploring Low-Cost Materials to Make Pattern-Based Lock-Screens Accessible for Users with Visual Impairments or Deafblindness , 2020, ASSETS.