Measuring Human Trust in a Virtual Assistant using Physiological Sensing in Virtual Reality

With the advancement of Artificial Intelligence technology to make smart devices, understanding how humans develop trust in virtual agents is emerging as a critical research field. Through our research, we report on a novel methodology to investigate user’s trust in auditory assistance in a Virtual Reality (VR) based search task, under both high and low cognitive load and under varying levels of agent accuracy. We collected physiological sensor data such as electroencephalography (EEG), galvanic skin response (GSR), and heart-rate variability (HRV), subjective data through questionnaire such as System Trust Scale (STS), Subjective Mental Effort Questionnaire (SMEQ) and NASA-TLX. We also collected a behavioral measure of trust (congruency of users’ head motion in response to valid/ invalid verbal advice from the agent). Our results indicate that our custom VR environment enables researchers to measure and understand human trust in virtual agents using the matrices, and both cognitive load and agent accuracy play an important role in trust formation. We discuss the implications of the research and directions for future work.

[1]  Robert Oostenveld,et al.  The five percent electrode system for high-resolution EEG and ERP measurements , 2001, Clinical Neurophysiology.

[2]  Koichi Kise,et al.  Mental State Analysis on Eyewear , 2018, UbiComp/ISWC Adjunct.

[3]  Neera Jain,et al.  Dynamic modeling of trust in human-machine interactions , 2017, 2017 American Control Conference (ACC).

[4]  Wioleta Szwoch,et al.  Emotion Recognition Using Physiological Signals , 2015, MIDI '15.

[5]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[6]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[7]  Pavlo D. Antonenko,et al.  Using Electroencephalography to Measure Cognitive Load , 2010 .

[8]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[9]  Angel Jiménez Molina,et al.  Using Psychophysiological Sensors to Assess Mental Workload During Web Browsing , 2018, Sensors.

[10]  Soo-Young Lee,et al.  A Preliminary Study on Human Trust Measurements by EEG for Human-Machine Interactions , 2015, HAI.

[11]  W. Klimesch.  EEG-alpha rhythms and memory processes. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[12]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[13]  Kevin Sweeney.  Motion Artifact Processing Techniquesfor Physiological Signals , 2013 .

[14]  Fang Chen,et al.  Galvanic skin response (GSR) as an index of cognitive load , 2007, CHI Extended Abstracts.

[15]  Fang Chen,et al.  Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment , 2015, CHI Extended Abstracts.

[16]  Blanca Hernández-Ortega.  The role of post-use trust in the acceptance of a technology: Drivers and consequences , 2011 .

[17]  Kai Kunze,et al.  Assessing hands-free interactions for VR using eye gaze and electromyography , 2018, Virtual Reality.

[18]  Mark R. Lehto,et al.  Foundations for an Empirically Determined Scale of Trust in Automated Systems , 2000 .

[19]  Sandra G. Hart,et al.  NASA Task Load Index (TLX). Volume 1.0; Paper and Pencil Package , 1986 .

[20]  Katarzyna Samson,et al.  Effects of Cognitive Load on Trusting Behavior – An Experiment Using the Trust Game , 2015, PloS one.

[21]  Bruce H. Thomas,et al.  Levity: A Virtual Reality System that Responds to Cognitive Load , 2018, CHI Extended Abstracts.

[22]  Daniel McDuff,et al.  COGCAM: Contact-free Measurement of Cognitive Stress During Computer Tasks with a Digital Camera , 2016, CHI.

[23]  Mohamed Abouelenien,et al.  Detecting Human Thermal Discomfort via Physiological Signals , 2017, PETRA.

[24]  Glyn Lawson,et al.  The Relationship Between Presence and Trust in Virtual Reality , 2016, ECCE.

[25]  Hao Chen,et al.  Exploring the design space for multi-sensory heart rate feedback in immersive virtual reality , 2017, OZCHI.

[26]  Daniel P Ferris,et al.  Effects of virtual reality high heights exposure during beam-walking on physiological stress and cognitive loading , 2018, PloS one.

[27]  Neera Jain,et al.  Real-Time Sensing of Trust in Human-Machine Interactions , 2016 .

[28]  Bernard Barber,et al.  The Logic and Limits of Trust , 1983 .

[29]  Desney S. Tan,et al.  Using a low-cost electroencephalograph for task classification in HCI research , 2006, UIST.

[30]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[31]  Andreas Dunser,et al.  Combining EEG with Pupillometry to Improve Cognitive Workload Detection , 2015, Computer.

[32]  John D. Murphy,et al.  Avatars, People, and Virtual Worlds: Foundations for Research in Metaverses , 2009, J. Assoc. Inf. Syst..

[33]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[34]  Nilanjan Sarkar,et al.  Cognitive Load Measurement in a Virtual Reality-Based Driving System for Autism Intervention , 2017, IEEE Transactions on Affective Computing.

[35]  Yang Wang,et al.  Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks , 2012, OZCHI.

[36]  Theo Vurdubakis,et al.  Chasing Shadows: Control, Virtuality and the Production of Trust , 2001 .

[37]  Valerie J. Gawron.  Human performance measures handbook , 2000 .

[38]  Izak Benbasat,et al.  Online Consumer Trust and Live Help Interfaces: The Effects of Text-to-Speech Voice and Three-Dimensional Avatars , 2005, Int. J. Hum. Comput. Interact..

[39]  Arindam Dey,et al.  Exploration of an EEG-Based Cognitively Adaptive Training System in Virtual Reality , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[40]  W. Kirchner.  Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.

[41]  Huosheng Hu,et al.  The Usefulness of Mean and Median Frequencies in Electromyography Analysis , 2012 .

[42]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Martin Breidt,et al.  Face reality: investigating the Uncanny Valley for virtual faces , 2010, SIGGRAPH ASIA.

[44]  Mark Billinghurst,et al.  In AI We Trust: Investigating the Relationship between Biosignals, Trust and Cognitive Load in VR , 2019, VRST.

[45]  T. Chartrand,et al.  The Chameleon Effect as Social Glue: Evidence for the Evolutionary Significance of Nonconscious Mimicry , 2003 .

[46]  A. Hamilton,et al.  Testing the relationship between mimicry, trust and rapport in virtual reality conversations , 2016, Scientific Reports.