Wearable Assistive Tactile Communication Interface Based on Integrated Touch Sensors and Actuators

This paper presents the design and fabrication of a wearable tactile communication interface with vibrotactile feedback for assistive communication. The interface is based on finger Braille, which is a simple and efficient tactile communication method used by deafblind people. It consists of a flexible piezoresistive sensor and a vibrotactile actuator integrated together and positioned at the index, middle and ring fingers of both hands to represent the six dots of Braille. The sensors were made using flexible piezoresistive material whereas the actuator utilizes electromagnetic principle by means of a flexible coil and a tiny NdFeB permanent magnet. Both were integrated to realize a Bluetooth-enabled tactile communication glove which enables deafblind people to communicate using Braille codes. The evaluation with 20 end-users (10 deafblind and 10 sighted and hearing person) of the tactile interface under standardized conditions demonstrated that users can feel and distinguish the vibration at frequencies ranging from 10Hz to 200Hz which is within the perceivable frequency range for the FA-II receptors. The results show that it took non-experts in Braille within 25s and 55s to send and receive words like “BEST” and “JOURNAL”, with an accuracy of ~75% and 68% respectively.

[1]  Minoru Asada,et al.  Flexible Tri-Axis Tactile Sensor Using Spiral Inductor and Magnetorheological Elastomer , 2018, IEEE Sensors Journal.

[2]  Satoshi Ohtsuka,et al.  The Introduction of Tele-Support System for Deaf-Blind People Using Body-Braille and a Mobile Phone , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[3]  Jurgen Kosel,et al.  Magnetic Tactile Sensor for Braille Reading , 2016, IEEE Sensors Journal.

[4]  L. J. Gorny,et al.  Large Displacement in Relaxor Ferroelectric Terpolymer Blend Derived Actuators Using Al Electrode for Braille Displays , 2015, Scientific reports.

[5]  Vincent Hayward,et al.  Large-Area Soft e-Skin: The Challenges Beyond Sensor Designs , 2019, Proceedings of the IEEE.

[6]  Yasuhiko Jimbo,et al.  Finger Braille Recognition System for Non-disabled People who Communicate with Deafblind People , 2007 .

[7]  Sharmistha Roy,et al.  SPARSHA: A Low Cost Refreshable Braille for Deaf-Blind People for Communication with Deaf-Blind and Non-disabled Persons , 2013, ICDCIT.

[8]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[9]  Hadi Heidari,et al.  Multisensory Smart Glove for Tactile Feedback in Prosthetic Hand , 2016 .

[10]  Ning Lan,et al.  Sensory information feedback for neural prostheses , 2020 .

[11]  P. König,et al.  Beyond sensory substitution—learning the sixth sense , 2005, Journal of neural engineering.

[12]  Richard E. Ladner,et al.  V-braille: haptic braille perception using a touch-screen and vibration on mobile phones , 2010, ASSETS '10.

[13]  M. Sile O'Modhrain,et al.  Refreshing Refreshable Braille Displays , 2015, IEEE Transactions on Haptics.

[14]  Chien-Hsing Chou,et al.  Portable communication aid for deaf blind people , 2001 .

[15]  Tanay Choudhary,et al.  A Braille-based mobile communication and translation glove for deaf-blind people , 2015, 2015 International Conference on Pervasive Computing (ICPC).

[16]  Á. Pascual-Leone,et al.  Tactile spatial resolution in blind Braille readers , 2000, Neurology.

[17]  Maria Chiara Carrozza,et al.  Haptic-assistive technologies for audition and vision sensory disabilities , 2018, Disability and rehabilitation. Assistive technology.

[18]  Joel C. Huegel,et al.  MyVox—Device for the communication between people: blind, deaf, deaf-blind and unimpaired , 2014, IEEE Global Humanitarian Technology Conference (GHTC 2014).

[19]  Hadi Heidari,et al.  At-Home Computer-Aided Myoelectric Training System for Wrist Prosthesis , 2016, EuroHaptics.

[20]  Olga Troynikov,et al.  Evaluation of Flexible Force Sensors for Pressure Monitoring in Treatment of Chronic Venous Disorders , 2017, Sensors.

[21]  T. Someya,et al.  Sheet-Type Braille Displays by Integrating Organic Field-Effect Transistors and Polymeric Actuators , 2007, IEEE Transactions on Electron Devices.

[22]  Alexander Russomanno,et al.  Evaluating Approaches to Rendering Braille Text on a High-Density Pin Display , 2018, IEEE Transactions on Haptics.

[23]  Silvestro Micera,et al.  A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback , 2018, Science Robotics.

[24]  Marion A. Hersh,et al.  SmartFingerBraille: A tactile sensing and actuation based communication glove for deafblind people , 2017, 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE).

[25]  Shu Gong,et al.  Embedding Pinhole Vertical Gold Nanowire Electronic Skins for Braille Recognition. , 2019, Small.

[26]  Hyuk-Jun Kwon,et al.  Braille dot display module with a PDMS membrane driven by a thermopneumatic actuator , 2009 .

[27]  Ravinder Dahiya,et al.  Fingerprint‐Enhanced Capacitive‐Piezoelectric Flexible Sensing Skin to Discriminate Static and Dynamic Tactile Stimuli , 2019, Adv. Intell. Syst..

[28]  Nikolaos G. Bourbakis,et al.  Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[29]  Marion Hersh,et al.  Learning technology and disability - Overcoming barriers to inclusion: Evidence from a multicountry study , 2019, Br. J. Educ. Technol..

[30]  Antonio Barrientos,et al.  Tactile-Sight: A Sensory Substitution Device Based on Distance-Related Vibrotactile Flow , 2013 .

[31]  Nicholas B Langhals,et al.  Providing a Sense of Touch to Prosthetic Hands , 2015, Plastic and reconstructive surgery.

[32]  P. Bach-y-Rita.  Tactile Sensory Substitution Studies , 2004, Annals of the New York Academy of Sciences.

[33]  Ravinder Dahiya,et al.  Prosthetic Hand with Biomimetic Tactile Sensing and Force Feedback , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[34]  Marion Hersh,et al.  Inductance-Based Flexible Pressure Sensor for Assistive Gloves , 2018, 2018 IEEE SENSORS.

[35]  Fabrice Labeau,et al.  Nanowire FET Based Neural Element for Robotic Tactile Sensing Skin , 2017, Front. Neurosci..

[36]  T. Someya,et al.  A 4 V Operation, Flexible Braille Display Using Organic Transistors, Carbon Nanotube Actuators, and Organic Static Random‐Access Memory , 2011 .

[37]  I. Sakuma,et al.  Finger Braille recognition system for people who communicate with deafblind people , 2008, 2008 IEEE International Conference on Mechatronics and Automation.

[38]  Á. Pascual-Leone,et al.  Tactile spatial resolution in blind Braille readers , 2000, Neurology.

[39]  Nobuyuki Sasaki,et al.  Helen Keller Phone—a communication system for deaf-blind people using Body-Braille and Skype , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[40]  Luís Carriço,et al.  UbiBraille: designing and evaluating a vibrotactile Braille-reading device , 2013, ASSETS.

[41]  L. Lorenzelli,et al.  Piezoelectric graphene field effect transistor pressure sensors for tactile sensing , 2018, Applied Physics Letters.