The Body Surface as a Communication System: The State of the Art after 50 Years

The suggestion that the body surface might be used as an additional means of presenting information to human-machine operators has been around in the literature for nearly 50 years. Although recent technological advances have made the possibility of using the body as a receptive surface much more realistic, the fundamental limitations on the human information processing of tactile stimuli presented across the body surface are, however, still largely unknown. This literature review provides an overview of studies that have attempted to use vibrotactile interfaces to convey information to human operators. The importance of investigating any possible central cognitive limitations (i.e., rather than the peripheral limitations, such as related to sensory masking, that were typically addressed in earlier research) on tactile processing for the most effective design of body interfaces is highlighted. The applicability of the constraints emerging from studies of tactile processing under conditions of unisensory (i.e., purely tactile) stimulus presentation, to more ecologically valid conditions of multisensory stimulation, is also discussed. Finally, the results obtained from recent studies of tactile information processing under conditions of multisensory stimulation are described, and their implications for haptic/tactile interface design elucidated.

[1]  F. A. Geldard Adventures in tactile literacy. , 1957 .

[2]  M. Sile O'Modhrain,et al.  Cutaneous Grooves: Composing for the Sense of Touch , 2002 .

[3]  J C Craig,et al.  Vibrotactile pattern perception: extraordinary observers. , 1977, Science.

[4]  L. Bernstein,et al.  Enhanced speechreading in deaf adults: can short-term training/practice close the gap for hearing adults? , 2001, Journal of speech, language, and hearing research : JSLHR.

[5]  R. T. Verrillo,et al.  The effect of neural density and contactor surround on vibrotactile sensation magnitude , 1972 .

[6]  Andy Cockburn,et al.  Multimodal feedback for the acquisition of small targets , 2005, Ergonomics.

[7]  Ronald A. Rensink Seeing, sensing, and scrutinizing , 2000, Vision Research.

[8]  W Janssen,et al.  Behavioural effects of driver support , 1993 .

[9]  Yon Visell,et al.  Tactile sensory substitution: Models for enaction in HCI , 2009, Interact. Comput..

[10]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[11]  C E Sherrick Touch as a communicative sense: introduction. , 1985, The Journal of the Acoustical Society of America.

[12]  Michael J. Wright,et al.  Limitations for Change Detection in Multiple Gabor Targets , 2000 .

[13]  Hunter G. Hoffman,et al.  Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[14]  Maryanne Martin,et al.  Attention to words in different modalities: four-channel presentation with physical and semantic selection. , 1980, Acta psychologica.

[15]  Y. Matsuoka,et al.  Robotics for surgery. , 1999, Annual review of biomedical engineering.

[16]  R. S. French The discrimination of dot patterns as a function of number and average separation of dots. , 1953, Journal of Experimental Psychology.

[17]  G. Recanzone,et al.  Progressive improvement in discriminative abilities in adult owl monkeys performing a tactile frequency discrimination task. , 1992, Journal of neurophysiology.

[18]  Serge Marchand Touch, Temperature, and Pain in Health and Disease: Mechanisms and Assessments , 1995 .

[19]  Hong Z. Tan,et al.  Dynamic and predictive links between touch and vision , 2002, Experimental Brain Research.

[20]  W. R. Garner,et al.  Reaction time as a measure of span of attention. , 1948, The Journal of psychology.

[21]  Cristy Ho,et al.  Tactile and Multisensory Spatial Warning Signals for Drivers , 2008, IEEE Transactions on Haptics.

[22]  Ian R Summers,et al.  A broadband tactile array on the fingertip. , 2002, The Journal of the Acoustical Society of America.

[23]  Vincent Hayward,et al.  Do It Yourself Haptics: Part II [Tutorial] , 2008, IEEE Robotics & Automation Magazine.

[24]  Stimulus Parameter Considerations and Individual Differences in Cutaneous Sensitivity to Electropulse Stimulation , 1966, Perceptual and motor skills.

[25]  Robert J. K. Jacob,et al.  The VR Scooter: Wind and Tactile Feedback Improve User Performance , 2006, 3D User Interfaces (3DUI'06).

[26]  N. Durlach,et al.  Tactual displays of consonant voicing to supplement speechreading , 2003 .

[27]  Paul Bach-y-Rita,et al.  Brain plasticity: ‘visual’ acuity of blind persons via the tongue , 2001, Brain Research.

[28]  Paul Bach-y-Rita,et al.  Tactile displays , 1995 .

[29]  Charles Spence,et al.  Response Requirements Modulate Tactile Spatial Congruency Evects for Example, given That the Body Can Adopt a Variety of Diverent Postures, a Tactile Sensation Arising , 2022 .

[30]  J C Stevens,et al.  Spatial acuity of the body surface over the life span. , 1996, Somatosensory & motor research.

[31]  C. Spence,et al.  Back-to-front: Improved tactile discrimination performance in the space you cannot see , 2006, Neuroscience Letters.

[32]  R L BROWN,et al.  Effects of Intense Noise on Processing of Cutaneous Information of Varying Complexity , 1965, Perceptual and motor skills.

[33]  L. Pringle,et al.  Haptic numerosity perception: effect of item arrangement. , 1988, The American journal of psychology.

[34]  A. A. Collins,et al.  Vibrotactile localization on the arm: Effects of place, space, and age , 2003, Perception & psychophysics.

[35]  E. L. Kaufman,et al.  The discrimination of visual number. , 1949, The American journal of psychology.

[36]  Tactile Communication of Speech , 2003 .

[37]  K. Schimrigk,et al.  The touch corpuscles of the plantar surface of the big toe. Histological and histometrical investigations with respect to age. , 1980, European neurology.

[38]  Gary E. Burnett,et al.  Ubiquitous computing within cars: designing controls for non-visual use , 2001, Int. J. Hum. Comput. Stud..

[39]  J. C. Bliss,et al.  A direct translation reading aid for the blind , 1966 .

[40]  Willis J. Tompkins,et al.  Sensory substitution for space gloves and for space robots , 1987 .

[41]  Berthold Färber,et al.  BREAKING UP OPEN-LOOP STEERING CONTROL ACTIONS: THE STEERING WHEEL AS AN ACTIVE CONTROL DEVICE , 1993 .

[42]  Hiroo Iwata,et al.  Array force display for hardness distribution , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[43]  E. Lechelt Temporal numerosity discrimination: intermodal comparisons revisited. , 1975, British journal of psychology.

[44]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[45]  C. Spence,et al.  The cost of expecting events in the wrong sensory modality , 2001, Perception & psychophysics.

[46]  F. Campbell,et al.  The Magic Number 4 ± 0: A New Look at Visual Numerosity Judgements , 1976, Perception.

[47]  G. Brindley “Seeing” with Skin , 1973, Nature.

[48]  H Godthelp,et al.  Intelligent accelerator: an element of driver support , 1993 .

[49]  H Pashler,et al.  Familiarity and visual change detection , 1988, Perception & psychophysics.

[50]  S Weinstein,et al.  Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality , 1968 .

[51]  Frank Steyvers Vision in Vehicles IV , 1993 .

[52]  C. Spence,et al.  Attentional limitations in processing sequentially presented vibrotactile targets , 2002, Perception & psychophysics.

[53]  A. Nakamura,et al.  Somatosensory Homunculus as Drawn by MEG , 1998, NeuroImage.

[54]  Joshua D. Hoffman,et al.  Collision warning design to mitigate driver distraction , 2004, CHI.

[55]  Nadine B. Sarter,et al.  Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains , 1999, Hum. Factors.

[56]  J C Craig,et al.  Vibrotactile pattern recognition and discrimination at several body sites , 1984, Perception & psychophysics.

[57]  Hui Tang,et al.  An oral tactile interface for blind navigation , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[58]  Hong Z. Tan,et al.  Using spatial vibrotactile cues to direct visual attention in driving scenes , 2005 .

[59]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[60]  P. König,et al.  Beyond sensory substitution—learning the sixth sense , 2005, Journal of neural engineering.

[61]  T. B. Posey,et al.  Numerosity Discrimination of Tactile Stimuli , 1976, Perceptual and motor skills.

[62]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[63]  C. Spence,et al.  When vision ‘extinguishes’ touch in neurologically-normal people: extending the Colavita visual dominance effect , 2008, Experimental Brain Research.

[64]  Ryan M. Traylor,et al.  A Haptic Back Display for Attentional and Directional Cueing , 2003 .

[65]  W. H. Levison,et al.  Evaluation of tactual displays for flight control , 1973 .

[66]  Robert E. Schlegel,et al.  Tactile Stimulation of the Human Head for Information Display , 1994 .

[67]  Lynette A. Jones,et al.  Tactile display and vibrotactile pattern recognition on the torso , 2006, Adv. Robotics.

[68]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[69]  Z. Pylyshyn,et al.  What enumeration studies can show us about spatial attention: evidence for limited capacity preattentive processing. , 1993, Journal of experimental psychology. Human perception and performance.

[70]  C. Gross,et al.  The clothing effect: Tactile neurons in the precentral gyrus do not respond to the touch of the familiar primate chair , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[71]  Lorna M. Brown,et al.  Tactile crescendos and sforzandos: applying musical techniques to tactile icon design , 2006, CHI Extended Abstracts.

[72]  E. Lechelt Spatial numerosity discrimination as contingent upon sensory and extrinsic factors , 1971 .

[73]  S. Ferber,et al.  Spatial awareness is a function of the temporal not the posterior parietal lobe , 2001, Nature.

[74]  J. Collins,et al.  Vibrating insoles and balance control in elderly people , 2003, The Lancet.

[75]  M. Sivak The Information That Drivers Use: Is it Indeed 90% Visual? , 1996, Perception.

[76]  Robert W. Lindeman,et al.  Towards full-body haptic feedback: the design and deployment of a spatialized vibrotactile feedback system , 2004, VRST '04.

[77]  C. Spence,et al.  Numerosity Judgments for Tactile Stimuli Distributed over the Body Surface , 2006, Perception.

[78]  Avery Comarow,et al.  Good vibrations , 1988, Nature.

[79]  R. Howe,et al.  Vibrotactile Feedback for Industrial Telemanipulators , 1997, Dynamic Systems and Control.

[80]  D. Mcnicol,et al.  Modality-Specific Differences in the Processing of Spatially, Temporally, and Spatiotemporally Distributed Information , 1994, Perception.

[81]  J. V. R. Prasad,et al.  Open Architecture for Helicopter Tactile Cueing Systems , 2005 .

[82]  Mark S. Sanders,et al.  Human factors in engineering and design, 6th ed. , 1987 .

[83]  R. Cholewiak,et al.  Vibrotactile localization on the abdomen: Effects of place and space , 2004, Perception & psychophysics.

[84]  Christoph Braun,et al.  Behavioral significance of input-dependent plasticity of human somatosensory cortex , 2003, Neuroreport.

[85]  J. B.,et al.  The Power of Numerical Discrimination , 1871, Nature.

[86]  Alex Pentland,et al.  Tactual displays for sensory substitution and wearable computers , 2005, SIGGRAPH Courses.

[87]  Hong Z. Tan,et al.  When visual transients impair tactile change detection: A novel case of crossmodal change blindness? , 2006, Neuroscience Letters.

[88]  Charles Spence,et al.  Do “mudsplashes” induce tactile change blindness? , 2007, Perception & psychophysics.

[89]  N. Lavie Distracted and confused?: Selective attention under load , 2005, Trends in Cognitive Sciences.

[90]  Heidi Castle,et al.  Tactile displays for enhanced performance and safety , 2004, SPIE Defense + Commercial Sensing.

[91]  Christine Lawson,et al.  An Analysis of Children's Numerical Difficulties with the Aid of a Dyscalculia Test Battery and a Presentation of Remedial Approaches to Facilitate Aspects of Numerical Development , 2008 .

[92]  A H Rupert An instrumentation solution for reducing spatial disorientation mishaps. , 2000, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[93]  Benjamin A. Clegg,et al.  Interruption Management: The Use of Attention-Directing Tactile Cues , 2005, Hum. Factors.

[94]  Leonidas Deligiannidis A Scooter-like Device for Navigation in Virtual Environments , 2005, CISST.

[95]  C. Frith,et al.  Neural correlates of change detection and change blindness , 2001, Nature Neuroscience.

[96]  Angus H. Rupert,et al.  The use of a tactile interface to convey position and motion perceptions , 1994 .

[97]  J.D. Harry,et al.  Balancing act [noise based sensory enhancement technology] , 2005, IEEE Spectrum.

[98]  C. Spence,et al.  Modality-specific auditory and visual temporal processing deficits , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[99]  P. Bach-y-Rita Tactile Sensory Substitution Studies , 2004, Annals of the New York Academy of Sciences.

[100]  Joseph DiVita,et al.  Verification of the Change Blindness Phenomenon While Managing Critical Events on a Combat Information Display , 2004, Hum. Factors.

[101]  von Haller B. Gilmer,et al.  Toward Cutaneous Electro-Pulse Communication , 1961 .

[102]  John W. Senders,et al.  THE ATTENTIONAL DEMAND OF AUTOMOBILE DRIVING , 1967 .

[103]  Andrew Simpson,et al.  Subitizing in Tactile Perception , 2006, Psychological science.

[104]  Allison M. Okamura,et al.  Reality-based models for vibration feedback in virtual environments , 2001 .

[105]  H S Vitense,et al.  Multimodal feedback: an assessment of performance and mental workload , 2003, Ergonomics.

[106]  P A Wilkins,et al.  Noise and accidents--a review. , 1982, The Annals of occupational hygiene.

[107]  Karon E. MacLean,et al.  Foundations of Transparency in Tactile Information Design , 2008, IEEE Transactions on Haptics.

[108]  S. Brown,et al.  Balancing act. , 1996, The Canadian nurse.

[109]  Hong Z. Tan,et al.  Tactile change detection , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[110]  Hendrik A. H. C. van Veen,et al.  Waypoint navigation with a vibrotactile waist belt , 2005, TAP.

[111]  ROBERT H. GIBSON,et al.  Electrical Stimulation of Pain and Touch Systems , 1963, Nature.

[112]  Torsten Schubert,et al.  Aging and input processing in dual-task situations. , 2004, Psychology and aging.

[113]  J M Weisenberger,et al.  The role of tactile aids in providing information about acoustic stimuli. , 1987, The Journal of the Acoustical Society of America.

[114]  R. Haber,et al.  Contemporary theory and research in visual perception , 1970 .

[115]  Influence of an Irrelevant Stimulus Attribute on Numerosity Judgments , 1965, Perceptual and motor skills.

[116]  Richard D. Gilson,et al.  Multi-modal information processing for visual workload relief , 1980 .

[117]  P. Haggard,et al.  Shared representations in body perception. , 2006, Acta psychologica.

[118]  B. B. Morgan,et al.  Masking of Cutaneous Sensations in Multiple Stimulus Presentations , 1965, Perceptual and motor skills.

[119]  Robert D. Howe,et al.  Tactile Display of Vibratory Information in Teleoperation and Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[120]  Charles Spence,et al.  Head orientation biases tactile localization , 2007, Brain Research.

[121]  Robert W. Kentridge,et al.  Detectability of onsets versus offsets in the change detection paradigm. , 2003, Journal of vision.

[122]  Wiel Janssen,et al.  In-Vehicle Collision Avoidance Support Under Adverse Visibility Conditions , 1997 .

[123]  C. Spence,et al.  The failure to detect tactile change: A tactile analogue of visual change blindness , 2006, Psychonomic bulletin & review.

[124]  Frank A. Saunders,et al.  A wearable tactile sensory aid for profoundly deaf children , 2005, Journal of Medical Systems.

[125]  Die grundsätze der gestaltwahrnehmung in der haptik , 1957 .

[126]  James C. Bliss,et al.  Optical-to-Tactile Image Conversion for the Blind , 1970 .

[127]  E C Lechelt,et al.  Pulse Number Discrimination in Tactile Spatio-Temporal Patterns , 1974, Perceptual and motor skills.

[128]  Jon Driver,et al.  Cross-modal links in attention between audition, vision, and touch: Implications for interface design , 1997 .

[129]  S. Weghorst,et al.  Virtual reality and tactile augmentation in the treatment of spider phobia: a case report. , 1997, Behaviour research and therapy.

[130]  F. A. Geldard,et al.  MULTIPLE CUTANEOUS STIMULATION: THE DISCRIMINATION OF VIBRATORY PATTERNS. , 1965, The Journal of the Acoustical Society of America.

[131]  Cristy Ho,et al.  Verbal interface design: Do verbal directional cues automatically orient visual spatial attention? , 2006, Comput. Hum. Behav..

[132]  Charles Spence,et al.  Crossmodal change blindness between vision and touch. , 2007, Acta psychologica.

[133]  Benjamin A. Clegg,et al.  Tactile interruption management: tactile cues as task-switching reminders , 2006, Cognition, Technology & Work.

[134]  C. Spence,et al.  The differential effect of vibrotactile and auditory cues on visual spatial attention , 2006, Ergonomics.

[135]  J. V. Van Erp,et al.  Presenting directions with a vibrotactile torso display , 2005, Ergonomics.

[136]  B. Rockstroh,et al.  Increased Cortical Representation of the Fingers of the Left Hand in String Players , 1995, Science.

[137]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[138]  Jan B. F. van Erp,et al.  Tactile Navigation Display , 2000, Haptic Human-Computer Interaction.

[139]  Ron Kupers,et al.  “Seeing” through the tongue: cross-modal plasticity in the congenitally blind , 2004 .

[140]  F. Campbell,et al.  The Dependence of the Visual Numerosity Limit on Orientation, Colour, and Grouping in the Stimulus , 1976, Perception.

[141]  Hong Z. Tan,et al.  Short term memory for tactile stimuli , 2008, Brain Research.

[142]  Hendrik A. H. C. van Veen,et al.  Tactile Information Presentation in the Cockpit , 2000, Haptic Human-Computer Interaction.

[143]  Carter C. Collins,et al.  Tactile Television - Mechanical and Electrical Image Projection , 1970 .

[144]  Michael Scott,et al.  Effect of cognitive load on tactor location identification in zero-g , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[145]  P. M. Rossini,et al.  Neuromagnetic somatosensory homunculus: A non-invasive approach in humans , 1991, Neuroscience Letters.

[146]  R W Cholewiak,et al.  The generation of vibrotactile patterns on a linear array: Influences of body site, time, and presentation mode , 2000, Perception & psychophysics.

[147]  THE EFFECT OF WARNING SIGNALS ON REACTION TIME IN YOUTH AND OLD AGE. , 1964, Journal of gerontology.

[148]  Hervé Segond,et al.  Human Spatial Navigation via a Visuo-Tactile Sensory Substitution System , 2005, Perception.

[149]  P. Bach-y-Rita Brain mechanisms in sensory substitution , 1972 .

[150]  Lorna M. Brown,et al.  Multidimensional tactons for non-visual information presentation in mobile devices , 2006, Mobile HCI.

[151]  J. V. Erp,et al.  Vibrotactile in-vehicle navigation system , 2004 .

[152]  Lorna M. Brown,et al.  Feel who's talking: using tactons for mobile phone alerts , 2006, CHI Extended Abstracts.

[153]  Charles Spence,et al.  Multisensory numerosity judgments for visual and tactile stimuli , 2007, Perception & psychophysics.

[154]  Steven S. Hsiao,et al.  Control and Pattern Specification for a High Density Tactile Array , 1998, Dynamic Systems and Control.

[155]  M Akamatsu,et al.  Please Scroll down for Article Ergonomics a Comparison of Tactile, Auditory, and Visual Feedback in a Pointing Task Using a Mouse-type Device , 2022 .

[156]  G. Vallar Extrapersonal Visual Unilateral Spatial Neglect and Its Neuroanatomy , 2001, NeuroImage.

[157]  Frank Ai Geldard Cutaneous coding of optical signals: the optohapt , 1966 .

[158]  Boris M. Velichkovsky,et al.  Change detection and occlusion modes in road-traffic scenarios , 2002 .

[159]  Hanfeng Yuan,et al.  Tactual display of consonant voicing to supplement lipreading , 2005, The Journal of the Acoustical Society of America.

[160]  John C Stevens,et al.  Tactile information transfer: a comparison of two stimulation sites. , 2005, The Journal of the Acoustical Society of America.

[161]  Frank Moss,et al.  Medical technology: Balancing the unbalanced , 2003, Nature.

[162]  Robert W. Lindeman,et al.  Effective Vibrotactile Cueing in a Visual Search Task , 2003, INTERACT.

[163]  Masahiko Inami,et al.  SmartTouch: electric skin to touch the untouchable , 2004, IEEE Computer Graphics and Applications.

[164]  Pradeep K. Khosla,et al.  Psychophysical Characterization and Testbed Validation of a Wearable Vibrotactile Glove for Telemanipulation , 2003, Presence: Teleoperators & Virtual Environments.

[165]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[166]  C. Spence Multisensory attention and tactile information-processing , 2002, Behavioural Brain Research.

[167]  D. Scott Perceptual learning. , 1974, Queen's nursing journal.

[168]  J. Grimes On the failure to detect changes in scenes across saccades. , 1996 .

[169]  Robert W. Lindeman,et al.  Handling of Virtual Contact in Immersive Virtual Environments: Beyond Visuals , 2002, Virtual Reality.

[170]  John Duncan,et al.  Restricted attentional capacity within but not between sensory modalities , 1997, Nature.

[171]  Amra Hodzic,et al.  Improvement and Decline in Tactile Discrimination Behavior after Cortical Plasticity Induced by Passive Tactile Coactivation , 2004, The Journal of Neuroscience.

[172]  B. Godde,et al.  Tactile Coactivation-Induced Changes in Spatial Discrimination Performance , 2000, The Journal of Neuroscience.

[173]  Hunter G. Hoffman,et al.  Physically touching and tasting virtual objects enhances the realism of virtual experiences , 1998, Virtual Reality.

[174]  J. Kirman Tactile communication of speech: a review and an analysis. , 1973, Psychological bulletin.

[175]  F A GELDARD,et al.  Some neglected possibilities of communication. , 1960, Science.

[176]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[177]  Abderrahmane Kheddar,et al.  Tactile interfaces: a state-of-the-art survey , 2004 .

[178]  R. D. Gilson,et al.  A kinesthetic-tactual display for stall deterrence , 1975 .

[179]  W. Penfield,et al.  SOMATIC MOTOR AND SENSORY REPRESENTATION IN THE CEREBRAL CORTEX OF MAN AS STUDIED BY ELECTRICAL STIMULATION , 1937 .

[180]  S Lakatos,et al.  Time—distance relations in shifting attention between locations on one’s body , 1997, Perception & psychophysics.