Automotive user interfaces for the support of non-driving-related activities

Driving a car has changed a lot since the first car was invented. Today, drivers do not only maneuver the car to their destination but also perform a multitude of additional activities in the car. This includes for instance activities related to assistive functions that are meant to increase driving safety and reduce the driver’s workload. However, since drivers spend a considerable amount of time in the car, they often want to perform non-driving-related activities as well. In particular, these activities are related to entertainment, communication, and productivity. The driver’s need for such activities has vastly increased, particularly due to the success of smart phones and other mobile devices. As long as the driver is in charge of performing the actual driving task, such activities can distract the driver and may result in severe accidents. Due to these special requirements of the driving environment, the driver ideally performs such activities by using appropriately designed in-vehicle systems. The challenge for such systems is to enable flexible and easily usable non-driving-related activities while maintaining and increasing driving safety at the same time. The main contribution of this thesis is a set of guidelines and exemplary concepts for automotive user interfaces that offer safe, diverse, and easy-to-use means to perform non-driving-related activities besides the regular driving tasks. Using empirical methods that are commonly used in human-computer interaction, we investigate various aspects of automotive user interfaces with the goal to support the design and development of future interfaces that facilitate non-driving-related activities. The first aspect is related to using physiological data in order to infer information about the driver’s workload. As a second aspect, we propose a multimodal interaction style to facilitate the interaction with multiple activities in the car. In addition, we introduce two concepts for the support of commonly used and demanded non-driving-related activities: For communication with the outside world, we investigate the driver’s needs with regard to sharing ride details with remote persons in order to increase driving safety. Finally, we present a concept of time-adjusted activities (e.g., entertainment and productivity) which enable the driver to make use of times where only little attention is required. Starting with manual, non-automated driving, we also consider the rise of automated driving modes.

[1]  Manfred Tscheligi,et al.  Are We There Yet? A Probing Study to Inform Design for the Rear Seat of Family Cars , 2011, INTERACT.

[2]  Eric Rodgman,et al.  The role of driver distraction in traffic crashes , 2001 .

[3]  Neville A. Stanton,et al.  In-Vehicle Information Systems to Meet the Needs of Drivers , 2011, Int. J. Hum. Comput. Interact..

[4]  Yvonne Barnard,et al.  UDRIVE: the European naturalistic driving study , 2014 .

[5]  Tatjana Streit,et al.  Deutsches Mobilitätspanel (MOP) – Wissenschaftliche Begleitung und Auswertungen Bericht 2013/2014: Alltagsmobilität und Fahrleistung , 2015 .

[6]  Fabrice Axisa,et al.  Flexible technologies and smart clothing for citizen medicine, home healthcare, and disease prevention , 2005, IEEE Transactions on Information Technology in Biomedicine.

[7]  A. Kramer,et al.  Physiological metrics of mental workload: A review of recent progress , 1990, Multiple-task performance.

[8]  Frank E. Pollick,et al.  Language-based multimodal displays for the handover of control in autonomous cars , 2015, AutomotiveUI.

[9]  Georg Jahn,et al.  Peripheral detection as a workload measure in driving: Effects of traffic complexity and route guidance system use in a driving study , 2005 .

[10]  David G. Curry In-Vehicle Cell Phones: Fatal Distraction—Yes or No? , 2001 .

[11]  Susan G. Hill,et al.  Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In A , 1989 .

[12]  Roland Bader,et al.  A study on user acceptance of proactive in-vehicle recommender systems , 2011, AutomotiveUI.

[13]  Eric Horvitz,et al.  Hang on a sec!: effects of proactive mediation of phone conversations while driving , 2011, CHI.

[14]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[15]  Johan Engström,et al.  Comparison between Visual and Tactile Signal Detection Tasks Applied to the Safety Assessment of In-Vehicle Information Systems , 2017 .

[16]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[17]  Albrecht Schmidt,et al.  MakeIt: Integrate User Interaction Times in the Design Process of Mobile Applications , 2008, Pervasive.

[18]  Yang Li,et al.  User-defined motion gestures for mobile interaction , 2011, CHI.

[19]  Stefania Serafin,et al.  Volvo intelligent news: A context aware multi modal proactive recommender system for in-vehicle use , 2014, Pervasive Mob. Comput..

[20]  Klaus Bengler,et al.  Gesture Control for Use in Automobiles , 2000, MVA.

[21]  Christian A. Müller,et al.  Geremin": 2D microgestures for drivers based on electric field sensing , 2011, IUI '11.

[22]  Andreas Butz,et al.  Mensch-Maschine-Interaktion , 2014 .

[23]  Spiros Dimolitsas SUBJECTIVE ASSESSMENT METHODS FOR THE MEASUREMENT OF DIGITAL SPEECH CODER QUALITY , 1993 .

[24]  Klaus Bengler,et al.  Measurement of Driver's Distraction for an Early Prove of Concepts in Automotive Industry at the Example of the Development of a Haptic Touchpad , 2011, HCI.

[25]  Alireza Sahami Shirazi,et al.  TaxiMedia: An Interactive Context-Aware Entertainment and Advertising System , 2009, GI Jahrestagung.

[26]  Bernhard Preim,et al.  Interaktive Systeme - Band 1: Grundlagen, Graphical User Interfaces, Informationsvisualisierung , 2010 .

[27]  Bryant Walker Smith,et al.  Automated and Autonomous Driving: Regulation under Uncertainty , 2015 .

[28]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[29]  Joshua E. Domeyer,et al.  Towards Operationalizing Driver Distraction , 2017 .

[30]  Omer Tsimhoni,et al.  Language pattern analysis for automotive natural language speech applications , 2010, AutomotiveUI.

[31]  Cyriel Diels,et al.  Self-driving carsickness. , 2016, Applied ergonomics.

[32]  alireza. sahami Towards an Application Platform That Allows Access to Sensors in the Car , .

[33]  Zhang Hua,et al.  Speech recognition interface design for in-vehicle system , 2010, AutomotiveUI.

[34]  W. D. Jones Building safer cars , 2002 .

[35]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[36]  Romeo Chua,et al.  Perceptual-Motor Interaction: Some Implications For Hci , 2007 .

[37]  Motoyuki Akamatsu,et al.  Automotive Technology and Human Factors Research: Past, Present, and Future , 2013 .

[38]  Paul Green,et al.  Safety and Usability of Speech Interfaces for In-Vehicle Tasks while Driving: A Brief Literature Review , 2006 .

[39]  Nora Broy Stereoscopic 3D user interfaces : exploring the potentials and risks of 3D displays in cars , 2016 .

[40]  Bruce N. Walker,et al.  Investigating a new display format for CarPlay to decrease impact of mode change inputs , 2015, AutomotiveUI.

[41]  A. H. Roscoe,et al.  Heart rate as a psychophysiological measure for in-flight workload assessment. , 1993, Ergonomics.

[42]  Thomas A. Dingus,et al.  An overview of the 100-car naturalistic study and findings , 2005 .

[43]  Thomas A. Dingus,et al.  THE 100 CAR NATURALISTIC DRIVING STUDY, PHASE I - EXPERIMENTAL DESIGN , 2002 .

[44]  Sara A. Bly,et al.  Quiet calls: talking silently on mobile phones , 2001, CHI.

[45]  Thomas E. Nygren,et al.  Psychometric Properties of Subjective Workload Measurement Techniques: Implications for Their Use in the Assessment of Perceived Mental Workload , 1991 .

[46]  Keith J. Burnham,et al.  A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction , 2007 .

[47]  Albrecht Schmidt,et al.  Shifting Gears: User Interfaces in the Age of Autonomous Driving , 2016, IEEE Pervasive Computing.

[48]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[49]  Albrecht Schmidt,et al.  A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions , 2016, CHI.

[50]  Alireza Sahami Shirazi,et al.  Enabling micro-entertainment in vehicles based on context information , 2010, AutomotiveUI.

[51]  Per Persson,et al.  DeDe: design and evaluation of a context-enhanced mobile messaging system , 2005, CHI.

[52]  Manfred Tscheligi,et al.  The car data toolkit: smartphone supported automotive HCI research , 2013, AutomotiveUI.

[53]  Mark E. Russell,et al.  Integrated automotive sensors , 2002 .

[54]  J. Dusek,et al.  Impact of Incremental Increases in Cognitive Workload on Physiological Arousal and Performance in Young Adult Drivers , 2009 .

[55]  Mohammed Yeasin,et al.  Speech-gesture driven multimodal interfaces for crisis management , 2003, Proc. IEEE.

[56]  Azzedine Boukerche,et al.  Vehicular Ad Hoc Networks: A New Challenge for Localization-Based Systems , 2008, Comput. Commun..

[57]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[58]  Vincent G. Duffy,et al.  Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement , 2007 .

[59]  Celine Pering Taming of the ring: context specific social mediation for communication devices , 2002, CHI Extended Abstracts.

[60]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[61]  Michael A. Regan,et al.  Driver distraction: A review of the literature , 2003 .

[62]  Gary E. Burnett On-the-Move and in Your Car: An Overview of HCI Issues for In-Car Computing , 2009, Int. J. Mob. Hum. Comput. Interact..

[63]  S. Lorkowski,et al.  Travel Time Measurements using GSM and GPS Probe Data , 2009 .

[64]  Sharon Oviatt,et al.  Multimodal Interfaces , 2008, Encyclopedia of Multimedia.

[65]  Johan Engström,et al.  Effects of visual and cognitive load in real and simulated motorway driving , 2005 .

[66]  Christian Wolff,et al.  Human modeling in a driver analyzing context: challenge and benefit , 2011, AutomotiveUI.

[67]  Stefan Mattes,et al.  Surrogate Distraction Measurement Techniques: The Lane Change Test , 2009 .

[68]  Markus Maurer,et al.  Rechtsfolgen zunehmender Fahrzeugautomatisierung , 2012 .

[69]  Eric Horvitz,et al.  Cars, calls, and cognition: investigating driving and divided attention , 2010, CHI.

[70]  Christian A. Müller,et al.  Multimodal Input in the Car, Today and Tomorrow , 2011, IEEE MultiMedia.

[71]  Brad A. Myers,et al.  Eyes on the road, hands on the wheel: thumb-based interaction techniques for input on steering wheels , 2007, GI '07.

[72]  Albrecht Schmidt,et al.  Exploring user expectations for context and road video sharing while calling and driving , 2013, AutomotiveUI.

[73]  Christina M. Rudin-Brown,et al.  BEHAVIOURAL ADAPTATION TO ADAPTIVE CRUISE CONTROL (ACC): IMPLICATIONS FOR PREVENTIVE STRATEGIES , 2004 .

[74]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[75]  Jason I. Hong,et al.  Undistracted driving: a mobile phone that doesn't distract , 2011, HotMobile '11.

[76]  Andrew L. Kun,et al.  Exploring the effects of visual cognitive load and illumination on pupil diameter in driving simulators , 2012, ETRA.

[77]  Andreas Riener,et al.  "FaceLight": potentials and drawbacks of thermal imaging to infer driver stress , 2012, AutomotiveUI.

[78]  Dot Hs,et al.  The 100 Car Naturalistic Driving Study , 2002 .

[79]  Amardeo Sarma,et al.  CAR-2-X Communication for Safety and Infotainment in Europe , 2008 .

[80]  Gary E. Burnett,et al.  An extended keystroke level model (KLM) for predicting the visual demand of in-vehicle information systems , 2007, CHI.

[81]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[82]  Changle Zhou,et al.  Real-Time Eye Detection in Video Streams , 2008, 2008 Fourth International Conference on Natural Computation.

[83]  Bastian Pfleging (Non-) Driving-Related Activities in the Car: Defining Driver Activities for Manual and Automated Driving , 2015 .

[84]  Jeff K Caird,et al.  A meta-analysis of the effects of cell phones on driver performance. , 2008, Accident; analysis and prevention.

[85]  Albrecht Schmidt,et al.  Gestural interaction on the steering wheel: reducing the visual demand , 2011, CHI.

[86]  B. J. Fogg,et al.  CommuterNews: a prototype of persuasive in-car entertainment , 2000, CHI Extended Abstracts.

[87]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[88]  A. H. Roscoe,et al.  A Subjective Rating Scale for Assessing Pilot Workload in Flight: A decade of Practical Use , 1990 .

[89]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982 .

[90]  Michael Rahier,et al.  Nutzerzentrierte Gestaltung und Entwicklung eines kontextsensitiven HMI , 2015, Mensch & Computer Workshopband.

[91]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[92]  Natasha Merat,et al.  Multisensory Signal Detection: How Does Driving and IVIS Management Affect Performance? , 2007 .

[93]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[94]  Gary Burnett,et al.  Defining Driver Distraction , 2005 .

[95]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[96]  Bryan Reimer,et al.  Defining workload in the context of driver state detection and HMI evaluation , 2012, AutomotiveUI.

[97]  Hamid Gharavi,et al.  Scanning Advanced Automobile Technology , 2007, Proceedings of the IEEE.

[98]  Andreas Butz,et al.  pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems , 2009, Mobile HCI.

[99]  Andrew L. Kun,et al.  Exploring the Influence of Light and Cognitive Load on Pupil Diameter Driving Simulator Studies , 2017 .

[100]  Bastian Pfleging,et al.  SpeeT: A Multimodal Interaction Style Combining Speech and Touch Interaction in Automotive Environments , 2011 .

[101]  Hans-Peter Krüger,et al.  Workload-Management im Verkehr: Prädiktion von Fahrerbeanspruchung durch Informationen in digitalen Karten , 2008 .

[102]  Albrecht Schmidt,et al.  Multimodal interaction in the car: combining speech and gestures on the steering wheel , 2012, AutomotiveUI.

[103]  N. Meshkati Heart Rate Variability and Mental Workload Assessment , 1988 .

[104]  F.R.H. Zijlstra,et al.  Efficiency in work behaviour: A design approach for modern tools , 1993 .

[105]  Malcolm Sabin,et al.  The Design Space , 2010 .

[106]  Dillon Funkhouser,et al.  Assessing Driver Distraction Due to In-Vehicle Video Systems through Field Testing at the Pecos Research and Testing Center , 2007 .

[108]  Elena Mugellini,et al.  WheelSense: Enabling Tangible Gestures on the Steering Wheel for In-Car Natural Interaction , 2013, HCI.

[109]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[110]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[111]  陈红松 Window cleaning device , 1998 .

[112]  Richard M Michaels EFFECT OF EXPRESSWAY DESIGN ON DRIVER TENSION RESPONSES , 1962 .

[113]  Olsson Measuring Driver Visual Distraction with a Peripheral Detection Task , 2000 .

[114]  C. A. Pickering,et al.  A Review of Automotive Human Machine Interface Technologies and Techniques to Reduce Driver Distraction , 2007 .

[115]  Alison Smiley,et al.  A Meta-Analysis of Driving Performance and Crash Risk Associated with the Use of Cellular Telephones while Driving , 2017 .