Live Biofeedback as a User Interface Design Element: A Review of the Literature

With the advances in sensor technology and real-time processing of neurophysiological data, a growing body of academic literature has begun to explore how live biofeedback can be integrated into information systems for everyday use. While researchers have traditionally studied live biofeedback in the clinical domain, the proliferation of affordable mobile sensor technology enables researchers and practitioners to consider live biofeedback as a user interface element in contexts such as decision support, education, and gaming. In order to establish the current state of research on live biofeedback, we conducted a literature review on studies that examine self and foreign live biofeedback based on neurophysiological data for healthy subjects in an information systems context. By integrating a body of highly fragmented work from computer science, engineering and technology, information systems, medical science, and psychology, this paper synthesizes results from existing research, identifies knowledge gaps, and suggests directions for future research. In this vein, this review can serve as a reference guide for researchers and practitioners on how to integrate self and foreign live biofeedback into information systems for everyday use.

[1]  Joel J. P. C. Rodrigues,et al.  Real time falls prevention and detection with biofeedback monitoring solution for mobile environments , 2013, 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services (Healthcom 2013).

[2]  Nilanjan Sarkar,et al.  Maintaining Optimal Challenge in Computer Games through Real-Time Physiological Feedback , 2005 .

[3]  Andrew N. Meltzoff,et al.  The neural bases of cooperation and competition: an fMRI investigation , 2004, NeuroImage.

[4]  N. Ravaja,et al.  Intragroup Emotions: Physiological Linkage and Social Presence , 2016, Front. Psychol..

[5]  Jeffrey Heer,et al.  Peripheral paced respiration: influencing user physiology during information work , 2011, UIST.

[6]  Daniel M. Hicks,et al.  Improving the Graphical User Interface (GUI) for the Dynamic Feedback Signal Set (DyFSS): Increasing Accessibility for the Neurodiverse , 2016 .

[7]  Torsten Wörtwein,et al.  Exploring feedback strategies to improve public speaking: an interactive virtual audience framework , 2015, UbiComp.

[8]  Sue C. Bratton,et al.  Enhancing Personal Wellness in Counselor Trainees Using Biofeedback: An Exploratory Study , 2001, Applied psychophysiology and biofeedback.

[9]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[10]  Andrew James Murphy,et al.  Communicating and using biomechanical measures through visual cues to optimise safe and effective rowing , 2016 .

[11]  Daniela Palomba,et al.  Improving Managers’ Psychophysical Well-Being: Effectiveness of Respiratory Sinus Arrhythmia Biofeedback , 2016, Applied psychophysiology and biofeedback.

[12]  G. Bonanno,et al.  Brief Report The coherence of emotion systems: Comparing “on‐line” measures of appraisal and facial expressions, and self‐report , 2004 .

[13]  Rosalind W. Picard The Galvactivator: A glove that senses and communicates skin conductivity , 2001 .

[14]  Yvonne de Kort,et al.  Fun and Sports: Enhancing the Home Fitness Experience , 2004, ICEC.

[15]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[16]  K. Yau,et al.  Interoception: the sense of the physiological condition of the body , 2003, Current Opinion in Neurobiology.

[17]  Christof Weinhardt,et al.  Integrating Biosignals into Information Systems: A NeuroIS Tool for Improving Emotion Regulation , 2013, J. Manag. Inf. Syst..

[18]  Holger Schnädelbach,et al.  ExoBuilding: breathing life into architecture , 2010, NordiCHI.

[19]  Abdulmotaleb El-Saddik,et al.  U-biofeedback: a multimedia-based reference model for ubiquitous biofeedback systems , 2013, Multimedia Tools and Applications.

[20]  Juan Carlos Augusto,et al.  Self-Aware Trader: A New Approach to Safer Trading , 2013, J. Univers. Comput. Sci..

[21]  Junichi Hoshino,et al.  A fitness game reflecting heart rate , 2006, ACE '06.

[22]  Eric Chance,et al.  Report on a Preliminary Study Using Breath Control and a Virtual Jogging Scenario as Biofeedback for Resilience Training , 2011, HCI.

[23]  R. Thayer Activation-Deactivation Adjective Check List: Current Overview and Structural Analysis , 1986 .

[24]  Alison Dillon Biofeedback and Gaming-Style Smartphone Applications as a Stress Reduction Intervention , 2018 .

[25]  Bodo Urban,et al.  AFFectIX - An Affective Component as Part of an E-Learning-System , 2007, HCI.

[26]  David M. Eagleman,et al.  Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput , 2015, Experimental Brain Research.

[27]  H. Ekanayake,et al.  Biofeedback based computational approach for Working Stress reduction through meditation technique , 2016, 2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer).

[28]  Weichung Joe Shih,et al.  Heart Rate Variability Biofeedback Increases Baroreflex Gain and Peak Expiratory Flow , 2003, Psychosomatic medicine.

[29]  Olle Hilborn,et al.  The Aiming Game: Using a Game with Biofeedback for Training in Emotion Regulation , 2011, DiGRA Conference.

[30]  Suleman Shahid,et al.  aMAZEd: designing an affective social game for children , 2007, IDC.

[31]  S. Reeves,et al.  Exploring skin conductance synchronisation in everyday interactions , 2014, NordiCHI.

[32]  Magnus Moar,et al.  'Ere be dragons: an interactive artwork , 2005, MULTIMEDIA '05.

[33]  René Riedl,et al.  On the biology of technostress: literature review and research agenda , 2012, DATB.

[34]  Rodrigo de Oliveira,et al.  TripleBeat: enhancing exercise performance with persuasion , 2008, Mobile HCI.

[35]  François Bernier,et al.  Using Biofeedback while Immersed in a Stressful Videogame Increases the Effectiveness of Stress Management Skills in Soldiers , 2012, PloS one.

[36]  Montserrat Conde Pastor,et al.  The Influence of Respiration on Biofeedback Techniques , 2008, Applied psychophysiology and biofeedback.

[37]  M. Greenberg,et al.  The Effects of Respiratory Sinus Arrhythmia Biofeedback on Heart Rate Variability and Posttraumatic Stress Disorder Symptoms: A Pilot Study , 2009, Applied psychophysiology and biofeedback.

[38]  Jérémy Frey,et al.  TOBE: Tangible Out-of-Body Experience , 2015, TEI.

[39]  B. Stein,et al.  The Merging of the Senses , 1993 .

[40]  Christof Weinhardt,et al.  Brownie: A Platform for Conducting NeuroIS Experiments , 2017, J. Assoc. Inf. Syst..

[41]  Luca Chittaro,et al.  Adaptation of Graphics and Gameplay in Fitness Games by Exploiting Motion and Physiological Sensors , 2007, Smart Graphics.

[42]  Loe M. G. Feijs,et al.  Design for Relaxation during Milk Expression Using Biofeedback , 2013, HCI.

[43]  M. Baciu,et al.  An fMRI study of the social competition in healthy subjects , 2011, Brain and Cognition.

[44]  Peter B. Shull,et al.  Haptic wearables as sensory replacement, sensory augmentation and trainer – a review , 2015, Journal of NeuroEngineering and Rehabilitation.

[45]  T. Kamarck,et al.  A global measure of perceived stress. , 1983, Journal of health and social behavior.

[46]  Olle Hilborn,et al.  A Serious Game using Physiological Interfaces for Emotion regulation Training in the Context of Financial Decision-Making , 2012, ECIS.

[47]  A. Tellegen,et al.  An alternative "description of personality": the big-five factor structure. , 1990, Journal of personality and social psychology.

[48]  Weixing Wang,et al.  Driver Fatigue Detection Based on Eye Tracking , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[49]  H Zeier,et al.  Arousal reduction with biofeedback-supported respiratory meditation , 1984, Biofeedback and self-regulation.

[50]  Rhodri Cusack,et al.  Listening to Your Heart , 2010, Psychological science.

[51]  G. Loewenstein Out of control: Visceral influences on behavior , 1996 .

[52]  Detlef Krömker,et al.  Zone of impulse: physiological data enhanced gaming , 2012, Mobile HCI.

[53]  J V Brady,et al.  Biofeedback heart rate training during exercise , 1977, Biofeedback and self-regulation.

[54]  Jason Bennett Thatcher,et al.  NeuroIS - Alternative or Complement to Existing Methods? Illustrating the Holistic Effects of Neuroscience and Self-Reported Data in the Context of Technostress Research , 2014, J. Assoc. Inf. Syst..

[55]  M. Snyder Self-monitoring of expressive behavior. , 1974 .

[56]  Jon Whittle,et al.  Biometric data sharing in the wild: Investigating the effects on online sports spectators , 2017, Int. J. Hum. Comput. Stud..

[57]  Lianying Ji,et al.  A Biofeedback System for Mobile Healthcare Based on Heart Rate Variability , 2013, ICSH.

[58]  R. Baumeister,et al.  Ego depletion: is the active self a limited resource? , 1998, Journal of personality and social psychology.

[59]  L. R. Goldberg The structure of phenotypic personality traits. , 1993, The American psychologist.

[60]  W. Boucsein Electrodermal activity, 2nd ed. , 2012 .

[61]  Holger Schnädelbach,et al.  ExoPranayama: a biofeedback-driven actuated environment for supporting yoga breathing practices , 2016, Personal and Ubiquitous Computing.

[62]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[63]  Stephen H. Fairclough,et al.  A research agenda for physiological computing , 2004, Interact. Comput..

[64]  Matthew R. Ebben,et al.  Moderating Laboratory Adaptation with the Use of a Heart-rate Variability Biofeedback Device (StressEraser®) , 2009, Applied psychophysiology and biofeedback.

[65]  Yoram Chisik,et al.  Space Connection - A Multiplayer Collaborative Biofeedback Game to Promote Empathy in Teenagers: A Feasibility Study , 2016, PhyCS.

[66]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[67]  Larissa Hjorth,et al.  TastyBeats: Celebrating Heart Rate Data with a Drinkable Spectacle , 2015, Tangible and Embedded Interaction.

[68]  Daniel A. Newman,et al.  Emotional intelligence: an integrative meta-analysis and cascading model. , 2010, The Journal of applied psychology.

[69]  Mary Czerwinski,et al.  MoodWings: a wearable biofeedback device for real-time stress intervention , 2013, PETRA '13.

[70]  John L. Andreassi Psychophysiology: Human behavior & physiological response, 5th ed. , 2007 .

[71]  Rebecca A. Grier,et al.  Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. , 2002, Emotion.

[72]  Mykola Pechenizkiy,et al.  What's Your Current Stress Level? Detection of Stress Patterns from GSR Sensor Data , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[73]  Scottie B. Reynolds,et al.  Biofeedback, relaxation training, and music: Homeostasis for coping with stress , 1984, Biofeedback and self-regulation.

[74]  Paul Johns,et al.  BioCrystal: An Ambient Tool for Emotion and Communication , 2015, Int. J. Mob. Hum. Comput. Interact..

[75]  Florian Mueller,et al.  Duel reality: a sword-fighting game for novel gameplay around intentionally hiding body data , 2013, IE '13.

[76]  Gilles Pourtois,et al.  Controlling the emotional heart: heart rate biofeedback improves cardiac control during emotional reactions. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[77]  O. John,et al.  Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. , 2003, Journal of personality and social psychology.

[78]  R. Bagozzi,et al.  On the nature and direction of relationships between constructs and measures. , 2000, Psychological methods.

[79]  Ryad Titah,et al.  Precision is in the Eye of the Beholder: Application of Eye Fixation-Related Potentials to Information Systems Research , 2014, J. Assoc. Inf. Syst..

[80]  Mark A. Fuller,et al.  Designing Interfaces with Social Presence: Using Vividness and Extraversion to Create Social Recommendation Agents , 2009, J. Assoc. Inf. Syst..

[81]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[82]  Richard S. Lazarus,et al.  Transactional theory and research on emotions and coping , 1987 .

[83]  Yu-Chun Huang,et al.  Heartbeat Jenga: A Biofeedback Board Game to Improve Coordination and Emotional Control , 2015, HCI.

[84]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[85]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[86]  Vincent J Monastra,et al.  The Effects of Stimulant Therapy, EEG Biofeedback, and Parenting Style on the Primary Symptoms of Attention-Deficit/Hyperactivity Disorder , 2002, Applied psychophysiology and biofeedback.

[87]  Mel Slater,et al.  SuperDreamCity: An Immersive Virtual Reality Experience That Responds to Electrodermal Activity , 2007, ACII.

[88]  T. C. Nicholas Graham,et al.  Heart rate control of exercise video games , 2009, Graphics Interface.

[89]  R.S.D. Wahidabanu,et al.  Robust R Peak and QRS detection in Electrocardiogram using Wavelet Transform , 2010 .

[90]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[91]  D. Shapiro,et al.  A review of biofeedback for mental disorders. , 1986, Hospital & community psychiatry.

[92]  René Riedl,et al.  Application Strategies for Neuroscience in Information Systems Design Science Research , 2013, J. Comput. Inf. Syst..

[93]  Angelika Dimoka,et al.  On the Use of Neuropyhsiological Tools in IS Research: Developing a Research Agenda for NeuroIS , 2012, MIS Q..

[94]  Dennis Majoe,et al.  Can the effectiveness of an online stress management program be augmented by wearable sensor technology , 2015 .

[95]  Erik Champion,et al.  Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback , 2007, DiGRA Conference.

[96]  Danielle Wilde,et al.  Displaying heart rate data on a bicycle helmet to support social exertion experiences , 2014, TEI '14.

[97]  Ian H. Robertson,et al.  Smartphone Applications Utilizing Biofeedback Can Aid Stress Reduction , 2016, Front. Psychol..

[98]  Holger Schnädelbach,et al.  ExoBuilding: Physiologically Driven Adaptive Architecture , 2012, TCHI.

[99]  Ryad Titah,et al.  Measure for Measure: A two study multi-trait multi-method investigation of construct validity in IS research , 2013, Comput. Hum. Behav..

[100]  Nathan F. Meier,et al.  Walking versus biofeedback: a comparison of acute interventions for stressed students , 2016, Anxiety, stress, and coping.

[101]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[102]  S. Berjot,et al.  Clinical stress assessment using a visual analogue scale. , 2012, Occupational medicine.

[103]  Athanasios V. Vasilakos,et al.  Affectively intelligent and adaptive car interfaces , 2010, Inf. Sci..

[104]  M. Dawson,et al.  The electrodermal system , 2007 .

[105]  Tamás D. Gedeon,et al.  Neuroscience and a Nomological Network for the Understanding and Assessment of Emotions in Information Systems Research , 2014, J. Manag. Inf. Syst..

[106]  Ryoko Ueoka,et al.  Development of the Horror Emotion Amplification System by Means of Biofeedback Method , 2015, HCI.

[107]  Diane Gromala,et al.  Sonic Cradle: designing for an immersive experience of meditation by connecting respiration to music , 2012, DIS '12.

[108]  Panos Markopoulos,et al.  HeartBeat: an outdoor pervasive game for children , 2009, CHI.

[109]  Mark Matthews,et al.  MoodLight: Exploring Personal and Social Implications of Ambient Display of Biosensor Data , 2015, CSCW.

[110]  Jennifer G. Sheridan,et al.  Jogging over a distance between Europe and Australia , 2010, UIST.

[111]  Alberto Botter,et al.  Young, Healthy Subjects Can Reduce the Activity of Calf Muscles When Provided with EMG Biofeedback in Upright Stance , 2016, Front. Physiol..

[112]  Raymond Chiong,et al.  Remote heart rate measurement using low-cost RGB face video: a technical literature review , 2018, Frontiers of Computer Science.

[113]  René Riedl,et al.  Design Blueprint for Stress-Sensitive Adaptive Enterprise Systems , 2017, Bus. Inf. Syst. Eng..

[114]  Stefan Rennick Egglestone,et al.  Breath control of amusement rides , 2011, CHI.

[115]  C. Spielberger,et al.  Manual for the State-Trait Anxiety Inventory , 1970 .

[116]  A. David,et al.  Biofeedback for Psychiatric Disorders: A Systematic Review , 2014, Applied Psychophysiology and Biofeedback.

[117]  Alan R. Hevner,et al.  Towards a NeuroIS Research Methodology: Intensifying the Discussion on Methods, Tools, and Measurement , 2014, J. Assoc. Inf. Syst..

[118]  Johannes Schöning,et al.  Investigating the effects of using biofeedback as visual stress indicator during video-mediated collaboration , 2014, CHI.

[119]  Perttu Hämäläinen,et al.  Shadow boxer: a physically interactive fitness game , 2004, NordiCHI '04.

[120]  Stephen Fairclough,et al.  Physiological data must remain confidential , 2014, Nature.

[121]  P. Hamilton,et al.  Open source ECG analysis , 2002, Computers in Cardiology.

[122]  Olle Hilborn,et al.  A Biofeedback Game for Training Arousal Regulation during a Stressful Task: The Space Investor , 2013, HCI.

[123]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[124]  Abdulmotaleb El-Saddik,et al.  Ubiquitous Biofeedback Serious Game for Stress Management , 2016, IEEE Access.

[125]  Alexandros Pantelopoulos,et al.  A survey on wearable biosensor systems for health monitoring , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[126]  Byron Reeves,et al.  Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player , 2010, Int. J. Hum. Comput. Stud..

[127]  Ionut Damian,et al.  Exploring the Potential of Realtime Haptic Feedback during Social Interactions , 2016, Tangible and Embedded Interaction.

[128]  Remko Soer,et al.  Increasing Performance of Professional Soccer Players and Elite Track and Field Athletes with Peak Performance Training and Biofeedback: A Pilot Study , 2016, Applied psychophysiology and biofeedback.

[129]  Shyamal Patel,et al.  A review of wearable sensors and systems with application in rehabilitation , 2012, Journal of NeuroEngineering and Rehabilitation.

[130]  Ivan Poupyrev,et al.  Biosignals as Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance , 2016, Conference on Designing Interactive Systems.

[131]  C. Kirschbaum,et al.  The 'Trier Social Stress Test'--a tool for investigating psychobiological stress responses in a laboratory setting. , 1993, Neuropsychobiology.

[132]  Ulrik R. Beierholm,et al.  Causal inference in perception , 2010, Trends in Cognitive Sciences.

[133]  Regan L. Mandryk,et al.  Biofeedback game design: using direct and indirect physiological control to enhance game interaction , 2011, CHI.

[134]  Thad Starner,et al.  BuzzWear: alert perception in wearable tactile displays on the wrist , 2010, CHI.

[135]  J. Gross,et al.  The tie that binds? Coherence among emotion experience, behavior, and physiology. , 2005, Emotion.

[136]  Angelika Dimoka,et al.  On the Foundations of NeuroIS: Reflections on the Gmunden Retreat 2009 , 2010, Commun. Assoc. Inf. Syst..

[137]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[138]  Luca Chittaro,et al.  Affective computing vs. affective placebo: Study of a biofeedback-controlled game for relaxation training , 2014, Int. J. Hum. Comput. Stud..

[139]  D. Kort,et al.  The Game Experience Questionnaire , 2013 .

[140]  Chiaki Nishimura,et al.  A learning model of autonomic function in biofeedback , 2007 .

[141]  Sigal G. Barsade,et al.  Human abilities: emotional intelligence. , 2008, Annual review of psychology.

[142]  Ricardo Gutierrez-Osuna,et al.  Dodging stress with a personalized biofeedback game , 2014, CHI PLAY.

[143]  Mark Matthews,et al.  Real-Time Representation Versus Response Elicitation in Biosensor Data , 2015, CHI.

[144]  Junichiro Hayano,et al.  Heart Rate Variability Biofeedback Improves Cardiorespiratory Resting Function During Sleep , 2013, Applied Psychophysiology and Biofeedback.

[145]  Geraldine Fitzpatrick,et al.  Understanding heart rate sharing: towards unpacking physiosocial space , 2012, CHI.

[146]  Juhani Heinilä,et al.  Fitness computer game with a bodily user interface , 2003, ICEC.

[147]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[148]  Jun Hu,et al.  LivingSurface: Biofeedback through Shape-changing Display , 2016, TEI.

[149]  E. Dale Walters,et al.  VOLUNTARY CONTROL OF INTERNAL STATES : PSYCHOLOGICAL AND PHYSIOLOGICAL 1 , 2003 .

[150]  Eva Hudlicka,et al.  Affective game engines: motivation and requirements , 2009, FDG.

[151]  Stefan Rennick Egglestone,et al.  Breathalising games: understanding the potential of breath control in game interfaces , 2011, Advances in Computer Entertainment Technology.

[152]  Perttu Hämäläinen,et al.  Using heart rate to control an interactive game , 2007, CHI.