Psychophysiological Responses to Media Interfaces

As advances in digital media technology continue to engage users in myriad ways, a critical question facing researchers of user experience is whether the human mind and body have evolved to meet such technological advancements? We answer this question in our lab by measuring users' psychophysiological responses (ECG, EEG, EDA, EOG and FEMG) to specific structural features of media interfaces such as interactivity and modality. As users engage interactive media, we measure their real-time physiological responses to careful variations in such stimuli as interactivity levels of news stories, pop-up advertisements, animation, and download speed, and correlate them with psychological responses pertaining to memory and attitudes. New interface features tend to engage the user's body in interesting new ways (e.g., Nintendo Wii), but they also pose new challenges to psychophysiological measurement (e.g., motion artifacts). We seek to brainstorm ways in which to meet these challenges and design more effective studies.

[1]  Norman O. Brown,et al.  Love's Body , 1966 .

[2]  R. Ketal,et al.  Affect, mood, emotion, and feeling: semantic considerations. , 1975, The American journal of psychiatry.

[3]  R. Goldstein,et al.  Attention to television: intrastimulus effects of movement and scene changes on alpha variation over time. , 1985, The International journal of neuroscience.

[4]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[5]  J. Cacioppo,et al.  Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. , 1986, Journal of personality and social psychology.

[6]  P. Tyson,et al.  Task-related stress and EEG alpha biofeedback , 1987, Biofeedback and self-regulation.

[7]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[8]  T W Picton,et al.  The P300 Wave of the Human Event‐Related Potential , 1992, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[9]  C. Izard,et al.  Four systems for emotion activation: cognitive and noncognitive processes. , 1993, Psychological review.

[10]  M. Lepper,et al.  Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. , 1993 .

[11]  Annie Lang,et al.  Measuring psychological responses to media messages , 1994 .

[12]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[13]  P. Lang International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[14]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[15]  Rosalind W. Picard Affective Computing , 1997 .

[16]  Bonnie E. John,et al.  The Strategic Use of Complex Computer Systems , 2000, Hum. Comput. Interact..

[17]  Elizabeth D. Mynatt,et al.  Nudge and shove: frequency thresholding for navigation in direct brain-computer interfaces , 2001, CHI Extended Abstracts.

[18]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  W. Keith Edwards,et al.  Intelligibility and Accountability: Human Considerations in Context-Aware Systems , 2001, Hum. Comput. Interact..

[20]  Pieter Desmet,et al.  Designing Products with Added Emotional Value: Development and Appllcation of an Approach for Research through Design , 2001 .

[21]  Kazuhiro Kuwabara,et al.  Connectedness oriented communication: fostering a sense of connectedness to augment social relationships , 2002, Proceedings 2002 Symposium on Applications and the Internet (SAINT 2002).

[22]  S. Sundar,et al.  The World Wide Wait: Exploring Physiological and Behavioral Effects of Download Speed , 2002 .

[23]  Anind K. Dey,et al.  Web accessibility for low bandwidth input , 2002, ASSETS.

[24]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[25]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[26]  Citations , 2002 .

[27]  M.M. Moore,et al.  Real-world applications for brain-computer interface technology , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Brad A. Myers,et al.  EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion , 2003, UIST '03.

[29]  R. Rauschenberger Attentional capture by auto- and allo-cues , 2003, Psychonomic bulletin & review.

[30]  T. Dalgleish Basic Emotions , 2004 .

[31]  Sriram Kalyanaraman,et al.  AROUSAL, MEMORY, AND IMPRESSION-FORMATION EFFECTS OF ANIMATION SPEED IN WEB ADVERTISING , 2004 .

[32]  Gary E. Birch,et al.  Brain-computer interface design for asynchronous control applications: improvements to the LF-ASD asynchronous brain switch , 2004, IEEE Transactions on Biomedical Engineering.

[33]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[34]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[35]  Veikko Surakka,et al.  Person-independent estimation of emotional experiences from facial expressions , 2005, IUI '05.

[36]  Heiner Stuckenschmidt,et al.  A personalized music system for motivation in sport performance , 2005, IEEE Pervasive Computing.

[37]  R.I.A. Mercuri,et al.  Technology as Experience , 2005, IEEE Transactions on Professional Communication.

[38]  Michael Beetz,et al.  Computerized real-time analysis of football games , 2005, IEEE Pervasive Computing.

[39]  Florian Michahelles,et al.  Sensing and monitoring professional skiers , 2005, IEEE Pervasive Computing.

[40]  Randy J. Pagulayan,et al.  A survey method for assessing perceptions of a game: The consumer playtest in game design , 2019, Game Stud..

[41]  Hatice Gunes,et al.  A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[42]  Kristina Höök,et al.  The sensual evaluation instrument: developing an affective evaluation tool , 2006, CHI.

[43]  Christopher K. Hsee,et al.  Decision and experience: why don't we choose what makes us happy? , 2006, Trends in Cognitive Sciences.

[44]  Kenneth R Boff,et al.  Revolutions and shifting paradigms in human factors & ergonomics. , 2006, Applied ergonomics.

[45]  M. Csíkszentmihályi,et al.  Experience Sampling Method: Measuring the Quality of Everyday Life , 2006 .

[46]  Regan L. Mandryk,et al.  Using psychophysiological techniques to measure user experience with entertainment technologies , 2006, Behav. Inf. Technol..

[47]  Wayne D. Gray,et al.  The soft constraints hypothesis: a rational analysis approach to resource allocation for interactive behavior. , 2006, Psychological review.

[48]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[49]  Desney S. Tan,et al.  Phosphor: explaining transitions in the user interface using afterglow effects , 2006, UIST.

[50]  Richard L. Hazlett,et al.  Measuring emotional valence during interactive experiences: boys at video game play , 2006, CHI.

[51]  Nadia Bianchi-Berthouze,et al.  Does Body Movement Engage You More in Digital Game Play? and Why? , 2007, ACII.

[52]  Jin Moen,et al.  From hand-held to body-worn: embodied experiences of the design and use of a wearable movement-based interaction concept , 2007, TEI.

[53]  Desney S. Tan,et al.  Playing with your brain: brain-computer interfaces and games , 2007, ACE '07.

[54]  Arthur C. Graesser,et al.  Toward an Affect-Sensitive AutoTutor , 2007, IEEE Intelligent Systems.

[55]  Sylvia D. Kreibig,et al.  Cardiovascular, electrodermal, and respiratory response patterns to fear- and sadness-inducing films. , 2007, Psychophysiology.

[56]  Russell Beauregard,et al.  User Experience Quality: A Conceptual Framework for Goal Setting and Measurement , 2007, HCI.

[57]  Karl J. Friston,et al.  Free-energy and the brain , 2007, Synthese.

[58]  D. Schacter,et al.  Remembering the past to imagine the future: the prospective brain , 2007, Nature Reviews Neuroscience.

[59]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Nadia Bianchi-Berthouze,et al.  Stirring up experience through movement in game play: effects on engagement and social behaviour , 2008, CHI.

[61]  Thomas Jürgensohn,et al.  Comparing Two Emotion Models for Deriving Affective States from Physiological Data , 2008, Affect and Emotion in Human-Computer Interaction.

[62]  Beverly Park Woolf,et al.  Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning , 2008 .

[63]  Michael Minge,et al.  Consideration of Multiple Components of Emotions in Human-Technology Interaction , 2008, Affect and Emotion in Human-Computer Interaction.

[64]  Shaowen Bardzell,et al.  Making user engagement visible: a multimodal strategy for interactive media experience research , 2008, CHI Extended Abstracts.

[65]  Jason Farquhar,et al.  Effects of Stimulus Type and of Error-Correcting Code Design on BCI Speller Performance , 2008, NIPS.

[66]  Changchun Liu,et al.  Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder , 2008, Int. J. Hum. Comput. Stud..

[67]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[68]  A. Braun,et al.  Neural Substrates of Spontaneous Musical Performance: An fMRI Study of Jazz Improvisation , 2008, PloS one.

[69]  Antonio Camurri,et al.  Technique for automatic emotion recognition by body gesture analysis , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[70]  Matthew S. Goodwin,et al.  Developing Innovative Technology for Future Personalized Autism Research and Treatment , 2008 .

[71]  Christos D. Katsis,et al.  Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[72]  Boris Reuderink,et al.  Games and Brain-Computer Interfaces: The State of the Art , 2008 .

[73]  Orit Shaer,et al.  Reality-based interaction: a framework for post-WIMP interfaces , 2008, CHI.

[74]  Georgios N. Yannakakis,et al.  Entertainment modeling through physiology in physical play , 2008, Int. J. Hum. Comput. Stud..

[75]  Krzysztof Z. Gajos,et al.  Improving the performance of motor-impaired users with automatically-generated, ability-based interfaces , 2008, CHI.

[76]  Gordon Ellis,et al.  Grand challenges for engineering , 2009, IEEE Engineering Management Review.

[77]  Jennifer A. Healey,et al.  Affect detection in the real world: Recording and processing physiological signals , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[78]  Anton Nijholt,et al.  Turning Shortcomings into Challenges: Brain-Computer Interfaces for Games , 2009, INTETAIN.

[79]  Gerhard Tröster,et al.  SwimMaster: a wearable assistant for swimmer , 2009, UbiComp.

[80]  Alessandro Canossa,et al.  Analyzing user behavior via gameplay metrics , 2009, Future Play.

[81]  John F. Canny,et al.  The Berkeley Tricorder: Ambulatory Health Monitoring , 2009, 2009 Sixth International Workshop on Wearable and Implantable Body Sensor Networks.

[82]  Anton Nijholt,et al.  Affective Pacman: A Frustrating Game for Brain-Computer Interface Experiments , 2009, INTETAIN.

[83]  H. Chad Lane,et al.  Pedagogical Experience Manipulation for Cultural Learning , 2009 .

[84]  Audrey Girouard Adaptive brain-computer interface , 2009, CHI Extended Abstracts.

[85]  Benton H. Calhoun,et al.  Body Area Sensor Networks: Challenges and Opportunities , 2009, Computer.

[86]  T. Baumgartner,et al.  Virtual Reality and the Role of the Prefrontal Cortex in Adults and Children , 2009, Frontiers in neuroscience.

[87]  Mark A. Neerincx,et al.  Modeling the Cognitive Task Load and Performance of Naval Operators , 2009, HCI.

[88]  Alessandro Canossa,et al.  Analyzing spatial user behavior in computer games using geographic information systems , 2009, MindTrek '09.

[89]  Kasia Muldner,et al.  Sensors Model Student Self Concept in the Classroom , 2009, UMAP.

[90]  Beverly Park Woolf,et al.  Affect-aware tutors: recognising and responding to student affect , 2009, Int. J. Learn. Technol..

[91]  D de Waard,et al.  Cardiovascular state changes during performance of a simulated ambulance dispatchers' task: potential use for adaptive support. , 2009, Applied ergonomics.

[92]  Egon L. van den Broek,et al.  Personalized affective music player , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[93]  Egon L. van den Broek,et al.  Guidelines for affective signal processing (ASP): From lab to life , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[94]  Kris Luyten,et al.  I Bet You Look Good on the Wall: Making the Invisible Computer Visible , 2009, AmI.

[95]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[96]  S M M Martens,et al.  Overlap and refractory effects in a brain–computer interface speller based on the visual P300 event-related potential , 2009, Journal of neural engineering.

[97]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[98]  Egon L. van den Broek,et al.  Unobtrusive Sensing of Emotions (USE) , 2009, J. Ambient Intell. Smart Environ..

[99]  G. Bieber,et al.  Emotion Recognition on the Go : Providing Personalized Services Based on Emotional States , 2009 .

[100]  J. Lindenberg,et al.  The MECA Project - Ontology-Based Data Portability for Space Missions , 2009, 2009 Third IEEE International Conference on Space Mission Challenges for Information Technology.

[101]  J.M. Bradshaw,et al.  Policy-Based Design of Human-Machine Collaboration in Manned Space Missions , 2009, 2009 Third IEEE International Conference on Space Mission Challenges for Information Technology.

[102]  Johan Eriksson,et al.  Effects of interactivity and 3D-motion on mental rotation brain activity in an immersive virtual environment , 2010, CHI.

[103]  F. Manganello Constructivist Instruction: Success or Failure? , 2010 .

[104]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[105]  Marc Hassenzahl,et al.  The Thing and I: Understanding the Relationship Between User and Product , 2005, Funology.