A Phonemic-Based Tactile Display for Speech Communication

Despite a long history of research, the development of synthetic tactual aids to support the communication of speech has proven to be a difficult task. The current paper describes a new tactile speech device based on the presentation of phonemic-based tactile codes. The device consists of 24 tactors under independent control for stimulation at the forearm. Using properties that include frequency and waveform of stimulation, amplitude, spatial location, and movement characteristics, unique tactile codes were designed for 39 consonant and vowel phonemes of the English language. The strategy for mapping the phonemes to tactile symbols is described, and properties of the individual phonemic codes are provided. Results are reported for an exploratory study of the ability of 10 young adults to identify the tactile symbols. The participants were trained to identify sets of consonants and vowels, before being tested on the full set of 39 tactile codes. The results indicate a mean recognition rate of 86 percent correct within one to four hours of training across participants. Thus, these results support the viability of a phonemic-based approach for conveying speech information through the tactile sense.

[1]  Nathaniel I. Durlach,et al.  Note on Information Transfer Rates in Human Communication , 1998, Presence.

[2]  Ryan M. Traylor,et al.  A Haptic Back Display for Attentional and Directional Cueing , 2003 .

[3]  J M Weisenberger,et al.  Evaluation of two multichannel tactile aids for the hearing impaired. , 1989, The Journal of the Acoustical Society of America.

[4]  S Weinstein,et al.  Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality , 1968 .

[5]  Siegfried Engelmann,et al.  Tactual Hearing Experiment with Deaf and Hearing Subjects , 1975 .

[6]  Alfred Mertins,et al.  Automatic speech recognition and speech variability: A review , 2007, Speech Commun..

[7]  Jonathan Tong,et al.  Two-Point Orientation Discrimination Versus the Traditional Two-Point Test for Tactile Spatial Acuity Assessment , 2013, Front. Hum. Neurosci..

[8]  T. D. Subash,et al.  Haptic Technology: A comprehensive review on its applications and future prospects , 2017 .

[9]  Lorraine A. Delhorne,et al.  Current Results of a Field Study of Adult Users of Tactile Aids , 1995 .

[10]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[11]  A Boothroyd,et al.  Tactile Presentation of Voice Fundamental Frequency as an Aid to the Speechreading of Sentences , 1988, Ear and hearing.

[12]  C M Reed,et al.  Reception of Morse code through motional, vibrotactile, and auditory stimulation , 1997, Perception & psychophysics.

[13]  L. Holt,et al.  Incidental categorization of spectrally complex non-invariant auditory stimuli in a computer game task. , 2005, The Journal of the Acoustical Society of America.

[14]  J M Weisenberger,et al.  Evaluation of a principal-components tactile aid for the hearing-impaired. , 1991, The Journal of the Acoustical Society of America.

[15]  H. Levitt.  Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[16]  S. Bolanowski,et al.  Four channels mediate the mechanical aspects of touch. , 1988, The Journal of the Acoustical Society of America.

[17]  Charlotte M. Reed,et al.  Research on Tactual Communication of Speech: Ideas, Issues, and Findings. , 1989 .

[18]  Slater E. Newman,et al.  Braille learning: Relative importance of seven variables† , 1987 .

[19]  J. Kirman.  Tactile communication of speech: a review and an analysis. , 1973, Psychological bulletin.

[20]  C M Reed,et al.  A study of the tactual and visual reception of fingerspelling. , 1990, Journal of speech and hearing research.

[21]  C M Reed,et al.  Research on tactile communication of speech: a review. , 1982, ASHA monographs.

[22]  C M Reed,et al.  A study of the tactual reception of sign language. , 1995, Journal of speech and hearing research.

[23]  W. M. Rabinowitz,et al.  Information transmission with a multifinger tactual display , 1999, Perception & psychophysics.

[24]  F. A. Geldard,et al.  The Cutaneous "Rabbit": A Perceptual Illusion , 1972, Science.

[25]  Ali Israr,et al.  Discrimination of Vowels with a Multi-finger Tactual Display , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[26]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[27]  Ali Israr,et al.  Control space of apparent haptic motion , 2011, 2011 IEEE World Haptics Conference.

[28]  C M Reed,et al.  Research on the Tadoma method of speech communication. , 1983, The Journal of the Acoustical Society of America.

[29]  Ali Israr,et al.  Tactile brush: drawing on skin with a tactile grid display , 2011, CHI.

[30]  Sethuraman Panchanathan,et al.  A haptic belt for vibrotactile communication , 2010, 2010 IEEE International Symposium on Haptic Audio Visual Environments and Games.

[31]  I. Pollack,et al.  The Information of Elementary Multidimensional Auditory Displays , 1954 .

[32]  L. Bernstein,et al.  Temporal and spatio-temporal vibrotactile displays for voice fundamental frequency: an initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearing-impaired individuals. , 1998, The Journal of the Acoustical Society of America.

[33]  N I Durlach,et al.  Tactile communication of speech: comparison of two computer-based displays. , 1988, Journal of rehabilitation research and development.

[34]  Eduardo E. Veas,et al.  Skin Reading: encoding text in a 6-channel haptic display , 2016, SEMWEB.

[35]  Evaristo J. Abril,et al.  Sensing Performance of a Vibrotactile Glove for Deaf-Blind People , 2017 .

[36]  A. Fraioli,et al.  Sensation magnitude of vibrotactile stimuli , 1969 .

[37]  David M. Eagleman,et al.  Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput , 2015, Experimental Brain Research.

[38]  J M Weisenberger,et al.  The Transmission of Phoneme‐Level Information by Multichannel Tactile Speech Perception Aids , 1995, Ear and hearing.

[39]  C M Reed,et al.  Preliminary results of speech-reception tests obtained with the synthetic Tadoma system. , 1988, Journal of rehabilitation research and development.

[40]  Hanfeng Yuan,et al.  Tactual display of consonant voicing to supplement lipreading , 2005, The Journal of the Acoustical Society of America.

[41]  Ali Israr,et al.  Coding Tactile Symbols for Phonemic Communication , 2018, CHI.

[42]  Ali Israr,et al.  A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin , 2018, EuroHaptics.