Can You Hear My Heartbeat?: Hearing an Expressive Biosignal Elicits Empathy

Interfaces designed to elicit empathy provide an opportunity for HCI with important pro-social outcomes. Recent research has demonstrated that perceiving expressive biosignals can facilitate emotional understanding and connection with others, but this work has been largely limited to visual approaches. We propose that hearing these signals will also elicit empathy, and test this hypothesis with sounding heartbeats. In a lab-based within-subjects study, participants (N = 27) completed an emotion recognition task in different heartbeat conditions. We found that hearing heartbeats changed participants’ emotional perspective and increased their reported ability to “feel what the other was feeling.” From these results, we argue that auditory heartbeats are well-suited as an empathic intervention, and might be particularly useful for certain groups and use-contexts because of its musical and non-visual nature. This work establishes a baseline for empathic auditory interfaces, and offers a method to evaluate the effects of future designs.

[1]  J. Clarkson,et al.  Inclusive design , 2010 .

[2]  O. Pollatos,et al.  Perceiving one's body shapes empathy , 2015, Physiology & Behavior.

[3]  Grace Leslie Inner Rhythms: Vessels as a Sustained Brain-Body Performance Practice , 2020, Leonardo.

[4]  E. Biddiss,et al.  Biomusic: A Novel Technology for Revealing the Personhood of People with Profound Multiple Disabilities , 2013, Augmentative and alternative communication.

[5]  B. Hermelin,et al.  Can children with autistic spectrum disorders perceive affect in music? An experimental investigation , 1999, Psychological Medicine.

[6]  Laura A. Dabbish,et al.  Supporting Social Interactions with an Expressive Heart Rate Sharing Application , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[7]  Istvan Molnar-Szakacs,et al.  Autism, Emotion Recognition and the Mirror Neuron System: The Case of Music , 2009, McGill journal of medicine : MJM : an international forum for the advancement of medical sciences by students.

[8]  Danielle Wilde,et al.  Displaying heart rate data on a bicycle helmet to support social exertion experiences , 2014, TEI '14.

[9]  Ivan Poupyrev,et al.  Biosignals as Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance , 2016, Conference on Designing Interactive Systems.

[10]  S. Lipscomb,et al.  Immersion in the virtual environment: the effect of a musical score on the video gaming experience. , 2004, Journal of physiological anthropology and applied human science.

[11]  David Rosenboom,et al.  Biofeedback and the Arts: Results of Early Experiments , 1989 .

[12]  Linda Krug,et al.  Toronto Empathy Questionnaire , 2016 .

[13]  Aleix M. Martinez Visual perception of facial expressions of emotion. , 2017, Current Opinion in Psychology.

[14]  S. Gosling,et al.  A very brief measure of the Big-Five personality domains , 2003 .

[15]  C. A. Baker,et al.  Meta-Analysis Reveals Adult Female Superiority in "Reading the Mind in the Eyes Test" , 2013 .

[16]  Davide Rocchesso,et al.  The Sonification Handbook , 2011 .

[17]  Valtteri Wikström,et al.  SynKin: A Game for Intentionally Synchronizing Biosignals , 2017, CHI Extended Abstracts.

[18]  Jennifer Healey,et al.  Affective wearables , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[19]  Kimiko Ryokai,et al.  Life-Affirming Biosensing in Public: Sounding Heartbeats on a Red Bench , 2019, CHI.

[20]  Anastasios A. Economides,et al.  Affective Learning: Empathetic Agents with Emotional Facial and Tone of Voice Expressions , 2012, IEEE Transactions on Affective Computing.

[21]  K. Ressler,et al.  Social cognition or social class and culture? On the interpretation of differences in social cognitive performance , 2019, Psychological Medicine.

[22]  Mark H. Davis,et al.  A Multidimensional Approach to Individual Differences in Empathy , 1980 .

[23]  R. Doherty The Emotional Contagion Scale: A Measure of Individual Differences , 1997 .

[24]  Laura A. Dabbish,et al.  The Effect of Expressive Biosignals on Empathy and Closeness for a Stigmatized Group Member , 2019, Proc. ACM Hum. Comput. Interact..

[25]  Xuemin Zhang,et al.  Increased accessibility to nonverbal communication through facial and expression recognition technologies for blind/visually impaired subjects , 2011, ASSETS.

[26]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[27]  Geraldine Fitzpatrick,et al.  Understanding heart rate sharing: towards unpacking physiosocial space , 2012, CHI.

[28]  Jessica R. Cauchard,et al.  Breeze: Sharing Biofeedback through Wearable Technologies , 2018, CHI.

[29]  Christian Mühl,et al.  Connecting Brains and Bodies: Applying Physiological Computing to Support Social Interaction , 2015, Interact. Comput..

[30]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[31]  Rory Allen,et al.  Autism, Music, and the Therapeutic Potential of Music in Alexithymia , 2010 .

[32]  Stephen W. Porges,et al.  The Polyvagal Theory: Neurophysiological Foundations of Emotions, Attachment, Communication, and Self-regulation , 2011 .

[33]  S. Preston,et al.  Empathy: Its ultimate and proximate bases. , 2001, The Behavioral and brain sciences.

[34]  E. Fombonne,et al.  The Effect of Music on Social Attribution in Adolescents with Autism Spectrum Disorders , 2009, Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence.

[35]  R. Hayward Empathy , 1971, The Lancet.

[36]  S. McAdams,et al.  Empathy and Emotional Contagion as a Link Between Recognized and Felt Emotions in Music Listening , 2013 .

[37]  H. Critchley,et al.  Interoception and emotion. , 2017, Current opinion in psychology.

[38]  P. Gomez,et al.  Relationships between musical structure and psychophysiological measures of emotion. , 2007, Emotion.

[39]  D. Levitin,et al.  The neurochemistry of music , 2013, Trends in Cognitive Sciences.

[40]  Ceenu George,et al.  Towards Augmenting IVR Communication with Physiological Sensing Data , 2019, CHI Extended Abstracts.

[41]  Fannie Liu Expressive Biosignals: Authentic Social Cues for Social Connection , 2019, CHI Extended Abstracts.

[42]  G. Bird,et al.  Theory of Mind Is Not Theory of Emotion: A Cautionary Note on the Reading the Mind in the Eyes Test , 2016, Journal of abnormal psychology.

[43]  B. H. Lo,et al.  Autism Spectrum Disorder , 2018, Journal of paediatrics and child health.

[44]  Elisabeth Hill,et al.  The Effects of Autism and Alexithymia on Physiological and Verbal Responsiveness to Music , 2013, Journal of autism and developmental disorders.

[45]  Kimiko Ryokai,et al.  Tensions of Data-Driven Reflection: A Case Study of Real-Time Emotional Biosensing , 2018, CHI.

[46]  Stefan Wiens,et al.  Heartbeat detection and the experience of emotions , 2000 .

[47]  E. Viding,et al.  The self to other model of empathy: Providing a new framework for understanding empathy impairments in psychopathy, autism, and alexithymia , 2014, Neuroscience & Biobehavioral Reviews.

[48]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[49]  J. O'Doherty,et al.  Empathy for Pain Involves the Affective but not Sensory Components of Pain , 2004, Science.

[50]  P. Juslin Musical Emotions Explained , 2019 .

[51]  J. Levinson Musical Expressiveness as Hearability‐as‐Expression , 2006 .

[52]  Azadeh Kushki,et al.  Biomusic: An Auditory Interface for Detecting Physiological Indicators of Anxiety in Children , 2016, Front. Neurosci..

[53]  Javier Jaimovich,et al.  Biosignal-driven Art: Beyond biofeedback , 2011 .

[54]  G. Bodie The Active-Empathic Listening Scale (AELS): Conceptualization and Evidence of Validity Within the Interpersonal Domain , 2011 .

[55]  Daniel Tranel,et al.  Cardiovascular and respiratory responses during musical mood induction. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[56]  K. Scherer Which Emotions Can be Induced by Music? What Are the Underlying Mechanisms? And How Can We Measure Them? , 2004 .

[57]  John C.-I. Chuang,et al.  Understanding Digitally-Mediated Empathy: An Exploration of Visual, Narrative, and Biosensory Informational Cues , 2019, CHI.

[58]  D. Neumann,et al.  Empathy-related responses to moving film stimuli depicting human and non-human animal targets in negative circumstances , 2008, Biological Psychology.

[59]  Jon Whittle,et al.  HeartLink: open broadcast of live biometric data to social networks , 2013, CHI.

[60]  Paweł W. Woźniak,et al.  HeartChat: Heart Rate Augmented Mobile Messaging to Support Empathy and Awareness , 2017 .

[61]  A. Cohen Music as a source of emotion in film , 2001 .

[62]  Joris H. Janssen A three-component framework for empathic technologies to augment human interaction , 2012, Journal on Multimodal User Interfaces.

[63]  Maja Pantic,et al.  Social signal processing: Survey of an emerging domain , 2009, Image Vis. Comput..

[64]  Z. Kövecses,et al.  Metaphor and Emotion: Language, Culture, and Body in Human Feeling , 2000 .

[65]  A. Gabrielsson,et al.  The role of structure in the musical expression of emotions , 2010 .

[66]  Riitta Parkkola,et al.  Is emotional contagion special? An fMRI study on neural systems for affective and cognitive empathy , 2008, NeuroImage.

[67]  Coye Cheshire,et al.  Habits of the Heart(rate): Social Interpretation of Biosignals in Two Interaction Contexts , 2016, GROUP.

[68]  M Boltz,et al.  Effects of background music on the remembering of filmed events , 1991, Memory & cognition.

[69]  Kevin Bird Within-subjects Designs , 2004 .

[70]  J. Decety,et al.  The functional architecture of human empathy. , 2004, Behavioral and cognitive neuroscience reviews.

[71]  S. Baron-Cohen,et al.  The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[72]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[73]  Mark H. Davis Empathy: A Social Psychological Approach , 1994 .

[74]  L. F. Barrett,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Interoceptive Sensitivity and Self-Reports of Emotional Experience , 2004 .

[75]  Marcelo M. Wanderley,et al.  Sonification of Emotion: Strategies and results from the intersection with music , 2014, Organised Sound.

[76]  Eva Hornecker,et al.  United-pulse: feeling your partner's pulse , 2008, Mobile HCI.

[77]  S. Baron-Cohen,et al.  Measuring empathy: reliability and validity of the Empathy Quotient , 2004, Psychological Medicine.

[78]  F. Happé,et al.  Do social and cognitive deficits curtail musical understanding? Evidence from autism and Down syndrome , 2008 .

[79]  D. Västfjäll,et al.  Emotional responses to music: the need to consider underlying mechanisms. , 2008, The Behavioral and brain sciences.

[80]  S. Baron-Cohen,et al.  Does the autistic child have a “theory of mind” ? , 1985, Cognition.

[81]  D. Grandjean,et al.  Rhythmic entrainment as a musical affect induction mechanism , 2017, Neuropsychologia.

[82]  E. Knowles,et al.  Social Class Predicts Emotion Perception and Perspective-Taking Performance in Adults , 2020, Personality & social psychology bulletin.

[83]  Wijnand A. IJsselsteijn,et al.  The Tell-Tale Heart: Perceived Emotional Intensity of Heartbeats , 2013, Int. J. Synth. Emot..

[84]  Tek-Jin Nam,et al.  Biosignal sharing for affective connectedness , 2014, CHI Extended Abstracts.

[85]  Wijnand A. IJsselsteijn,et al.  Intimate Heartbeats: Opportunities for Affective Communication Technology , 2010, IEEE Transactions on Affective Computing.

[86]  Annabel J. Cohen,et al.  Effects of Musical Soundtracks on Attitudes toward Animated Geometric Figures , 1988 .

[87]  P. Mckeever,et al.  Revealing Personhood Through Biomusic of Individuals Without Communicative Interaction Ability , 2011, Augmentative and alternative communication.

[88]  Daniel Müllensiefen,et al.  The Musicality of Non-Musicians: An Index for Assessing Musical Sophistication in the General Population , 2014, PloS one.

[89]  Laura A. Dabbish,et al.  Animo: Sharing Biosignals on a Smartwatch for Lightweight Social Connection , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[90]  R. Corcoran,et al.  The QCAE: A Questionnaire of Cognitive and Affective Empathy , 2009, European Psychiatry.

[91]  William A. Cunningham,et al.  Interpersonal Resonance: Developing Interpersonal Biofeedback for the Promotion of Empathy and Social Entrainment , 2018, Advances in Human Factors in Wearable Technologies and Game Design.

[92]  A. Tanaka,et al.  The Body as Musical Instrument , 2018, The Oxford Handbook of Music and the Body.

[93]  Fabio Zambetta,et al.  Neo-Noumena: Augmenting Emotion Communication , 2020, CHI.

[94]  James McCartney,et al.  SuperCollider, a New Real Time Synthesis Language , 1996, ICMC.

[95]  Rossio Motta-Ochoa,et al.  Participatory design of biomusic with users on the autism spectrum , 2019, 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).

[96]  T. Singer,et al.  The Social Neuroscience of Empathy , 2009, Annals of the New York Academy of Sciences.