Vibrotactile Displays: A Survey With a View on Highly Automated Driving

The task of car driving is automated to an ever greater extent. In the foreseeable future, drivers will no longer be required to touch the steering wheel and pedals and could engage in non-driving tasks such as working or resting. Vibrotactile displays have the potential to grab the attention of the driver when the automation reaches its functional limits and the driver has to take over control. The aim of the present literature survey is to outline the key physiological and psychophysical aspects of vibrotactile sensation and to provide recommendations and relevant research questions regarding the use of vibrotactile displays for taking over control from an automated vehicle. Results showed that a distinction can be made between four dimensions for coding vibrotactile information (amplitude, frequency, timing, and location), each of which can be static or dynamic. There is a consensus that frequency and amplitude are less suitable for coding information than location and timing. Vibrotactile stimuli have been shown to be effective as simple warnings. However, vibrations can evoke annoyance, and providing vibrations in close spatial-temporal proximity might cause a lack of comprehension of the signal. We describe the sequential stages of a take-over process and argue that vibrotactile displays are a promising candidate for redirecting the attention of a distracted driver. Furthermore, vibrotactile displays hold potential for supporting cognitive processing and action selection while resuming control of an automated vehicle. Finally, we argue that multimodal feedback should be used to assist the driver in the take-over process.

[1]  Nadine B. Sarter,et al.  The Need for Multisensory Interfaces in Support of Effective Attention Allocation in Highly Dynamic Event-Driven Domains: The Case of Cockpit Automation , 2000 .

[2]  René van Paassen,et al.  Design of a Haptic Gas Pedal for Active Car-Following Support , 2011, IEEE Transactions on Intelligent Transportation Systems.

[3]  Jan B. F. van Erp,et al.  A Tactile Seat for Direction Coding in Car Driving: Field Evaluation , 2009, IEEE Transactions on Haptics.

[4]  John B. Morrell,et al.  Design and evaluation of a vibrotactile seat to improve spatial awareness while driving , 2010, 2010 IEEE Haptics Symposium.

[5]  Peter A. Hancock,et al.  Fatigue and Automation-Induced Impairments in Simulated Driving Performance , 1998 .

[6]  W. E. Hick Quarterly Journal of Experimental Psychology , 1948, Nature.

[7]  Lynette A. Jones,et al.  Tactile Displays: Guidance for Their Design and Application , 2008, Hum. Factors.

[8]  Per-Anders Oskarsson,et al.  Evaluation of Tactile Drift Displays in Helicopter , 2014, HCI.

[9]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[10]  Mark Mulder,et al.  Sharing Control With Haptics , 2012, Hum. Factors.

[11]  Klaus Bengler,et al.  A transforming steering wheel for highly automated cars , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[12]  S. Bolanowski,et al.  A four-channel analysis of the tactile sensitivity of the fingertip: frequency selectivity, spatial summation, and temporal summation , 2002, Somatosensory & motor research.

[13]  Robert Gray,et al.  A Comparison of Tactile, Visual, and Auditory Warnings for Rear-End Collision Prevention in Simulated Driving , 2008, Hum. Factors.

[14]  Charles Spence,et al.  Tactile warning signals for in-vehicle systems. , 2015, Accident; analysis and prevention.

[15]  Vincent Hayward,et al.  Do it yourself haptics: part I , 2007, IEEE Robotics & Automation Magazine.

[16]  R W Cholewiak,et al.  Spatial factors in the perceived intensity of vibrotactile patterns. , 1979, Sensory processes.

[17]  Mark Mulder,et al.  The Effect of Haptic Support Systems on Driver Performance: A Literature Survey , 2015, IEEE Transactions on Haptics.

[18]  A. Fraioli,et al.  Sensation magnitude of vibrotactile stimuli , 1969 .

[19]  J. Hahn Vibrotactile adaptation and recovery measured by two methods. , 1966, Journal of experimental psychology.

[20]  S. A. Birrell,et al.  Vibrotactile pedals: provision of haptic feedback to support economical driving , 2013, Ergonomics.

[21]  Hong Z. Tan,et al.  Using spatial vibrotactile cues to direct visual attention in driving scenes , 2005 .

[22]  J. V. Van Erp,et al.  Presenting directions with a vibrotactile torso display , 2005, Ergonomics.

[23]  J.B.F. van Erp,et al.  Tactile Torso Display as Countermeasure to Reduce Night Vision Goggles Induced Drift , 2003 .

[24]  Georg v. Békésy,et al.  Synchronism of Neural Discharges and Their Demultiplication in Pitch Perception on the Skin and in Hearing , 1959 .

[25]  S. J. Bolanowski,et al.  Some characteristics of tactile channels , 2004, Behavioural Brain Research.

[26]  Natasha Merat,et al.  Transition to manual: driver behaviour when resuming control from a highly automated vehicle , 2014 .

[27]  R. T. Verrillo,et al.  Effect of Contactor Area on the Vibrotactile Threshold , 1963 .

[28]  Cristy Ho,et al.  Dynamic vibrotactile warning signals for frontal collision avoidance: towards the torso versus towards the head , 2015, Ergonomics.

[29]  Klaus Bengler,et al.  Taking Over Control from Highly Automated Vehicles , 2014 .

[30]  Berthold Färber,et al.  A Meta-Analysis of the Effects of Haptic Interfaces on Task Performance with Teleoperation Systems , 2013, IEEE Transactions on Haptics.

[31]  Thomas A. Dingus,et al.  Driver comprehension of multiple haptic seat alerts intended for use in an integrated collision avoidance system , 2011 .

[32]  Seungmoon Choi,et al.  Evaluating the effectiveness of haptic feedback on a steering wheel for BSW , 2010 .

[33]  Ralph Bruder,et al.  Driver assistance via optical information with spatial reference , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[34]  S. Bolanowski,et al.  Four channels mediate the mechanical aspects of touch. , 1988, The Journal of the Acoustical Society of America.

[35]  Robert J. Kosinski,et al.  A Literature Review on Reaction Time Kinds of Reaction Time Experiments , 2012 .

[36]  Hendrik A. H. C. van Veen,et al.  A Tactile Cockpit Instrument Supports the Control of Self-Motion During Spatial Disorientation , 2006, Hum. Factors.

[37]  S. Bolanowski,et al.  Hairy skin: psychophysical channels and their physiological substrates. , 1994, Somatosensory & motor research.

[38]  Daniel V. McGehee,et al.  Effects of Adaptive Cruise Control and Alert Modality on Driver Performance , 2009 .

[39]  Mario J. Enriquez,et al.  A pneumatic tactile alerting system for the driving environment , 2001, PUI '01.

[40]  J. V. Erp,et al.  Vibrotactile in-vehicle navigation system , 2004 .

[41]  Max Mulder,et al.  The Effect of Concurrent Bandwidth Feedback on Learning the Lane-Keeping Task in a Driving Simulator , 2011, Hum. Factors.

[42]  M. Duistermaat,et al.  Chapter 4 – Tactile Actuator Technology 4.1 Electrical , 2008 .

[43]  Brian M. Kleiner,et al.  Toward Developing an Approach for Alerting Drivers to the Direction of a Crash Threat , 2007, Hum. Factors.

[44]  Topi Kaaresoja,et al.  Perception of short tactile pulses generated by a vibration motor in a mobile phone , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[45]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[46]  Carryl L. Baldwin,et al.  Effect of Tactile Location, Pulse Duration, and Interpulse Interval on Perceived Urgency , 2014 .

[47]  Linda R. Elliott,et al.  Tactile Cuing to Augment Multisensory Human-Machine Interaction , 2015 .

[48]  F. A. Geldard,et al.  The Cutaneous "Rabbit": A Perceptual Illusion , 1972, Science.

[49]  R Parasuraman,et al.  Designing automation for human use: empirical studies and quantitative models , 2000, Ergonomics.

[50]  Braden J. McGrath,et al.  Tactile Situation Awareness System Flight Demonstration , 2004 .

[51]  Frederik Diederichs,et al.  Take-Over Requests for Automated Driving , 2015 .

[52]  Roope Raisamo,et al.  Delivering directional haptic cues through eyeglasses and a seat , 2015, 2015 IEEE World Haptics Conference (WHC).

[53]  Lorna M. Brown,et al.  A first investigation into the effectiveness of Tactons , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[54]  In Lee,et al.  Efficacy of haptic blind spot warnings applied through a steering wheel or a seatbelt , 2013 .

[55]  Rob Gray,et al.  A Comparison of Different Informative Vibrotactile Forward Collision Warnings: Does the Warning Need to Be Linked to the Collision Event? , 2014, PloS one.

[56]  Neville A. Stanton,et al.  Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence , 2014 .

[57]  Fanxing Meng,et al.  Dynamic Vibrotactile Signals for Forward Collision Avoidance Warning Systems , 2015, Hum. Factors.

[58]  Klaus Augsburg,et al.  DO DRIVERS STEER TOWARD OR AWAY FROM LATERAL DIRECTIONAL VIBRATIONS AT THE STEERING WHEEL ? , 2010 .

[59]  Jean-Michel Hoc,et al.  Lateral Control Assistance for Car Drivers: A Comparison of Motor Priming and Warning Systems , 2007, Hum. Factors.

[60]  Håkan Jansson,et al.  An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system , 2003 .

[61]  Hong Z. Tan,et al.  Driver Reaction Time to Tactile and Auditory Rear-End Collision Warnings While Talking on a Cell Phone , 2009, Hum. Factors.

[62]  Hendrik A. H. C. van Veen,et al.  Touch down: The effect of artificial touch cues on orientation in microgravity , 2006, Neuroscience Letters.

[63]  Alexander Toet,et al.  Uni-, bi- and tri-modal warning signals: effects of temporal parameters and sensory modality on perceived urgency , 2015 .

[64]  Thomas Maier,et al.  Driver Support by a Vibrotactile Seat Matrix – Recognition, Adequacy and Workload of Tactile Patterns in Take-over Scenarios During Automated Driving☆ , 2015 .

[65]  Cristy Ho,et al.  Tactile and Multisensory Spatial Warning Signals for Drivers , 2008, IEEE Transactions on Haptics.

[66]  Mehrdad Hosseini Zadeh,et al.  Active blind spot crash avoidance system: A haptic solution to blind spot collisions , 2010, 2010 IEEE International Symposium on Haptic Audio Visual Environments and Games.

[67]  Hong Z. Tan,et al.  To Go or Not to Go: Stimulus-Response Compatibility for Tactile and Auditory Pedestrian Collision Warnings , 2009, IEEE Transactions on Haptics.

[68]  F A GELDARD,et al.  Some neglected possibilities of communication. , 1960, Science.

[69]  Klaus Bengler,et al.  How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving , 2014 .

[70]  J.B.F. van Erp,et al.  Guidelines for the use of vibro-tactile displays in human computer interaction , 2002 .

[71]  Omer Tsimhoni,et al.  Haptic seat for automated driving: preparing the driver to take control effectively , 2015, AutomotiveUI.

[72]  Anil K. Raj,et al.  Vibrotactile Displays for Improving Spatial Awareness , 2000 .

[73]  Thomas K. Ferris,et al.  When Content Matters: The Role of Processing Code in Tactile Display Design , 2010, IEEE Transactions on Haptics.

[74]  Charles A. Green,et al.  Human Factors Issues Associated with Limited Ability Autonomous Driving Systems: Drivers’ Allocation of Visual Attention to the Forward Roadway , 2017 .

[75]  George A. Gescheider,et al.  Vibrotactile masking: Effects of oneand two-site stimulation , 1983, Perception & psychophysics.

[76]  Linda R. Elliott,et al.  A Meta-Analysis of Vibrotactile and Visual Information Displays for Improving Task Performance , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[77]  遠山 正彌,et al.  人体の解剖生理学 = Human anatomy & physiology , 2010 .

[78]  Naohisa Hashimoto,et al.  Measurement of Driver's Reaction Time to Failure of Steering Controller during Automatic Driving , 2005 .

[79]  Fabrice Vienne,et al.  Evaluation of human-machine cooperation applied to lateral control in car driving , 2006 .

[80]  Yong Gu Ji,et al.  Haptic perceptions in the vehicle seat , 2011 .

[81]  J. Craig Difference threshold for intensity of tactile stimuli , 1972 .

[82]  S J Bolanowski,et al.  The effects of aging on information-processing channels in the sense of touch: I. Absolute sensitivity. , 1994, Somatosensory & motor research.

[83]  Kathrin Zeeb,et al.  What determines the take-over time? An integrated model approach of driver take-over after automated driving. , 2015, Accident; analysis and prevention.

[84]  Brigitte Röder,et al.  Lost in the move? Secondary task performance impairs tactile change detection on the body , 2010, Consciousness and Cognition.

[85]  Cristy Ho,et al.  Looming Auditory and Vibrotactile Collision Warning for Safe Driving , 2017 .

[86]  Mark Mulder,et al.  Haptic shared control: smoothly shifting control authority? , 2012, Cognition, Technology & Work.

[87]  Klaus Bengler,et al.  “Take over!” How long does it take to get the driver back into the loop? , 2013 .

[88]  S J Bolanowski,et al.  Vibrotactile frequency for encoding a speech parameter. , 1977, The Journal of the Acoustical Society of America.

[89]  Cristy Ho,et al.  Reorienting Driver Attention with Dynamic Tactile Cues , 2014, IEEE Transactions on Haptics.

[90]  D. Begg A 2050 vision for London: what are the implications of driverless transport? , 2014 .

[91]  Hendrik A. H. C. van Veen,et al.  Waypoint navigation with a vibrotactile waist belt , 2005, TAP.