Using Noninvasive Brain Measurement to Explore the Psychological Effects of Computer Malfunctions on Users during Human-Computer Interactions

In today's technologically driven world, there is a need to better understand the ways that common computer malfunctions affect computer users. These malfunctions may have measurable influences on computer user's cognitive, emotional, and behavioral responses. An experiment was conducted where participants conducted a series of web search tasks while wearing functional near-infrared spectroscopy (fNIRS) and galvanic skin response sensors. Two computer malfunctions were introduced during the sessions which had the potential to influence correlates of user trust and suspicion. Surveys were given after each session to measure user's perceived emotional state, cognitive load, and perceived trust. Results suggest that fNIRS can be used to measure the different cognitive and emotional responses associated with computer malfunctions. These cognitive and emotional changes were correlated with users' self-report levels of suspicion and trust, and they in turn suggest future work that further explores the capability of fNIRS for the measurement of user experience during human-computer interactions.

[1]  J. E. Ball,et al.  Modeling the Emotional State of Computer Users , 1999 .

[2]  M. Ingvar,et al.  Distinct Contributions of the Dorsolateral Prefrontal and Orbitofrontal Cortex during Emotion Regulation , 2012, PloS one.

[3]  Dylan D. Schmorrow,et al.  Quantifying human information processing , 2005 .

[4]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[5]  Robert J. K. Jacob,et al.  This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy , 2011, CHI.

[6]  Judee K. Burgoon,et al.  Cognitive Biases and Nonverbal Cue Availability in Detecting Deception , 2008 .

[7]  S. Bunce,et al.  fNIR spectroscopy as a measure of cognitive task load , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[8]  Lisa Feldman Barrett,et al.  Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies , 2008, NeuroImage.

[9]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[10]  J. Russell.  A circumplex model of affect. , 1980 .

[11]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines , 2009, UIST '09.

[12]  Raja Parasuraman,et al.  Complacency and Bias in Human Use of Automation: An Attentional Integration , 2010, Hum. Factors.

[13]  M. Tomasello,et al.  Does the chimpanzee have a theory of mind? 30 years later , 2008, Trends in Cognitive Sciences.

[14]  Robert J. K. Jacob,et al.  Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy , 2009, CHI.

[15]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition) , 2004 .

[16]  Daniel J. McAllister.  Affect- and Cognition-Based Trust as Foundations for Interpersonal Cooperation in Organizations , 1995 .

[17]  B. Chance,et al.  Cognition-activated low-frequency modulation of light absorption in human brain. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J. R. Larson,et al.  Research strategies and tactics in industrial and organizational psychology. , 1990 .

[19]  R. Lewicki,et al.  Trust And Distrust: New Relationships and Realities , 1998 .

[20]  Angelika Dimoka,et al.  What Does the Brain Tell Us About Trust and Distrust? Evidence from a Functional Neuroimaging Study , 2010, MIS Q..

[21]  Elisabeth André,et al.  Exploiting emotions to disambiguate dialogue acts , 2004, IUI '04.

[22]  Bill McEvily,et al.  Measuring trust in organisational research: Review and recommendations , 2011 .

[23]  John A. R. Caldwell,et al.  Biomonitoring for Physiological and Cognitive Performance during Military Operations , 2005 .

[24]  Masako Okamoto,et al.  Virtual spatial registration of stand-alone fNIRS data to MNI space , 2007, NeuroImage.

[25]  Rosalind W. Picard,et al.  A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity , 2010, IEEE Transactions on Biomedical Engineering.

[26]  D. Yves von Cramon,et al.  The functional neuroanatomy of human working memory revisited Evidence from 3-T fMRI studies using classical domain-specific interference tasks , 2003, NeuroImage.

[27]  Leanne M. Hirshfield,et al.  The Construct of State-Level Suspicion , 2013, Hum. Factors.

[28]  E. Thompson.  Development and Validation of an Internationally Reliable Short-Form of the Positive and Negative Affect Schedule (PANAS) , 2007 .

[29]  Allen L. Edwards,et al.  The Relationship Between the Judged Desirability of a Trait and the Probability That the Trait Will Be Endorsed , 1953 .

[30]  Hoi-Chung Leung,et al.  Load response functions in the human spatial working memory circuit during location memory updating , 2007, NeuroImage.

[31]  R. Simons,et al.  Error-related psychophysiology and negative affect , 2004, Brain and Cognition.

[32]  Desney S. Tan,et al.  Feasibility and pragmatics of classifying working memory load with an electroencephalograph , 2008, CHI.

[33]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[34]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[35]  Alastair M. Hull,et al.  Neuroimaging findings in post-traumatic stress disorder , 2002, British Journal of Psychiatry.

[36]  James B. Stiff,et al.  Truth Biases and Aroused Suspicion in Relational Deception , 1992 .

[37]  Joseph B. Lyons,et al.  Trustworthiness and IT Suspicion: An Evaluation of the Nomological Network , 2011, Hum. Factors.

[38]  Robert J. K. Jacob,et al.  Combining Electroencephalograph and Functional Near Infrared Spectroscopy to Explore Users' Mental Workload , 2009, HCI.

[39]  T. Egner,et al.  Emotional processing in anterior cingulate and medial prefrontal cortex , 2011, Trends in Cognitive Sciences.

[40]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[41]  Eric Bergman,et al.  Introduction to Human Factors , 2012, Journal of diabetes science and technology.

[42]  Birsen Yazici,et al.  Human performance assessment using fNIR , 2005, SPIE Defense + Commercial Sensing.

[43]  Ben Shneiderman,et al.  Designing trust into online experiences , 2000, CACM.

[44]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[45]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004, Hum. Factors.

[46]  N. Kriegeskorte,et al.  Neural correlates of trust , 2007, Proceedings of the National Academy of Sciences.

[47]  Sungho Tak,et al.  NIRS-SPM: Statistical parametric mapping for near-infrared spectroscopy , 2009, NeuroImage.

[48]  J. Russell,et al.  Neural systems subserving valence and arousal during the experience of induced emotions. , 2010, Emotion.

[49]  J. Jonides,et al.  Storage and executive processes in the frontal lobes. , 1999, Science.

[50]  Robert J. K. Jacob,et al.  Distinguishing Difficulty Levels with Non-invasive Brain Activity Measurements , 2009, INTERACT.

[51]  Daphne N. Yu,et al.  High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. , 1997, Cerebral cortex.

[52]  Rosalind W. Picard.  Affective Computing , 1997 .

[53]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .

[54]  Yili Liu,et al.  Introduction to Human Factors Engineering (2nd Edition) , 2003 .

[55]  Elliot T. Berkman,et al.  Using Neuroscience to Broaden Emotion Regulation: Theoretical and Methodological Considerations. , 2009, Social and personality psychology compass.

[56]  Robert T. Knight,et al.  The role of lateral orbitofrontal cortex in the inhibitory control of emotion , 2006 .

[57]  Christopher D. Wickens,et al.  An introduction to human factors engineering , 1997 .

[58]  Colin Camerer,et al.  Not So Different After All: A Cross-Discipline View Of Trust , 1998 .

[59]  J. Gee,et al.  Early Stress Is Associated with Alterations in the Orbitofrontal Cortex: A Tensor-Based Morphometry Investigation of Brain Structure and Behavioral Risk , 2010, The Journal of Neuroscience.

[60]  J. H. Davis,et al.  The effect of the performance appraisal system on trust for management: A field quasi-experiment. , 1999 .