Human Computer Interaction Meets Psychophysiology: A Critical Perspective

Human computer interaction (HCI) groups are more and more often exploring the utility of new, lower cost electroencephalography (EEG) interfaces for assessing user engagement and experience as well as for directly controlling computers. While the potential benefits of using EEG are considerable, we argue that research is easily driven by what we term naive neurorealism. That is, data obtained with psychophysiological devices have poor reliability and uncertain validity, making inferences on mental states difficult. This means that unless sufficient care is taken to address the inherent shortcomings, the contributions of psychophysiological human computer interaction are limited to their novelty value rather than bringing scientific advance. Here, we outline the nature and severity of the reliability and validity problems and give practical suggestions for HCI researchers and reviewers on the way forward, and which obstacles to avoid. We hope that this critical perspective helps to promote good practice in the emerging field of psychophysiology in HCI.

[1]  A. Raz,et al.  Training the brain: Fact and fad in cognitive and behavioral remediation , 2012, Brain and Cognition.

[2]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[3]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[4]  Martin J. McKeown,et al.  Removing electroencephalographic artifacts: comparison between ICA and PCA , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[5]  Peder Olesen Larsen,et al.  The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index , 2010, Scientometrics.

[6]  D. Linden The P300: Where in the Brain Is It Produced and What Does It Tell Us? , 2005, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[7]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[8]  Gary H. Glover,et al.  A quantitative comparison of NIRS and fMRI across multiple cognitive tasks , 2011, NeuroImage.

[9]  A. Lutz,et al.  Long-term meditators self-induce high-amplitude gamma synchrony during mental practice. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[10]  H. Berger Über das Elektrenkephalogramm des Menschen , 1929, Archiv für Psychiatrie und Nervenkrankheiten.

[11]  J. Kounios,et al.  EEG Neurofeedback: A Brief Overview and an Example of Peak Alpha Frequency Training for Cognitive Enhancement in the Elderly , 2007, The Clinical neuropsychologist.

[12]  Kazimierz Karbowski,et al.  Hans Berger (1873–1941) , 2002, Journal of Neurology.

[13]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  R. Poldrack The role of fMRI in Cognitive Neuroscience: where do we stand? , 2008, Current Opinion in Neurobiology.

[15]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[16]  Klaus-Robert Müller,et al.  Introduction to machine learning for brain imaging , 2011, NeuroImage.

[17]  Bernhard Hommel,et al.  Enhancing cognitive control through neurofeedback: A role of gamma-band activity in managing episodic retrieval , 2010, NeuroImage.

[18]  H. Moldofsky,et al.  A spectral method for removing eye movement artifacts from the EEG. , 1978, Electroencephalography and clinical neurophysiology.

[19]  R. Poldrack Can cognitive processes be inferred from neuroimaging data? , 2006, Trends in Cognitive Sciences.

[20]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[21]  Zachary P. Infantolino,et al.  Issues in localization of brain function: The case of lateralized frontal cortex in cognition, emotion, and psychopathology , 2013, Front. Integr. Neurosci..

[22]  Marco Ferrari,et al.  A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application , 2012, NeuroImage.

[23]  S. Debener,et al.  How about taking a low-cost, small, and wireless EEG for a walk? , 2012, Psychophysiology.

[24]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[25]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[26]  Gilles Sicard,et al.  Functional Anatomy of Perceptual and Semantic Processing for Odors , 1999, Journal of Cognitive Neuroscience.

[27]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[28]  B. Rockstroh,et al.  Mapping EEG-potentials on the surface of the brain: A strategy for uncovering cortical sources , 2005, Brain Topography.

[29]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[30]  Stephen H. Fairclough,et al.  Measuring task engagement as an input to physiological computing , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[31]  Lei Zhou,et al.  Application of Frontal EEG Asymmetry to User Experience Research , 2014, HCI.

[32]  S. Muthukumaraswamy High-frequency brain activity and muscle artifacts in MEG/EEG: a review and recommendations , 2013, Front. Hum. Neurosci..

[33]  T. Robbins,et al.  Choosing between Small, Likely Rewards and Large, Unlikely Rewards Activates Inferior and Orbital Prefrontal Cortex , 1999, The Journal of Neuroscience.

[34]  Shaohan Hu,et al.  NeuroPhone: brain-mobile phone interface using a wireless EEG headset , 2010, MobiHeld '10.

[35]  J. E. Korteling,et al.  Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls , 2015, Front. Neurosci..

[36]  J. Duncan An adaptive coding model of neural function in prefrontal cortex , 2001 .

[37]  R. Pascual-Marqui Review of methods for solving the EEG inverse problem , 1999 .

[38]  E. Whitham,et al.  Scalp electrical recording during paralysis: Quantitative evidence that EEG frequencies above 20Hz are contaminated by EMG , 2007, Clinical Neurophysiology.

[39]  T. Hothorn,et al.  Domain-Based Benchmark Experiments: Exploratory and Inferential Analysis , 2016 .

[40]  David B Pisoni,et al.  PET imaging of differential cortical activation by monaural speech and nonspeech stimuli , 2002, Hearing Research.

[41]  Scott Makeig,et al.  Neuroelectromagnetic Forward Head Modeling Toolbox , 2010, Journal of Neuroscience Methods.

[42]  F. Ferri,et al.  Neural correlates of worry in generalized anxiety disorder and in normal controls: a functional MRI study , 2009, Psychological Medicine.

[43]  Deena Skolnick Weisberg,et al.  The Seductive Allure of Neuroscience Explanations , 2008, Journal of Cognitive Neuroscience.

[44]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[45]  W. Walter,et al.  Contingent Negative Variation : An Electric Sign of Sensori-Motor Association and Expectancy in the Human Brain , 1964, Nature.