EMMA: An Emotion-Aware Wellbeing Chatbot

The delivery of mental health interventions via ubiquitous devices has shown much promise. A conversational chatbot is a promising oracle for delivering appropriate just-in-time interventions. However, designing emotionally-aware agents, specially in this context, is under-explored. Furthermore, the feasibility of automating the delivery of just-in-time mHealth interventions via such an agent has not been fully studied. In this paper, we present the design and evaluation of EMMA (EMotion-Aware mHealth Agent) through a two-week long human-subject experiment with N=39 participants. EMMA provides emotionally appropriate micro-activities in an empathetic manner. We show that the system can be extended to detect a user's mood purely from smartphone sensor data. Our results show that our personalized machine learning model was perceived as likable via self-reports of emotion from users. Finally, we provide a set of guidelines for the design of emotion-aware bots for mHealth.

[1]  Konrad Paul Kording,et al.  Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study , 2015, Journal of medical Internet research.

[2]  Robert Morris,et al.  Crowdsourcing mental health and emotional well-being , 2015 .

[3]  Alex Pentland,et al.  Reality mining: sensing complex social systems , 2006, Personal and Ubiquitous Computing.

[4]  J. M. Digman.  PERSONALITY STRUCTURE: EMERGENCE OF THE FIVE-FACTOR MODEL , 1990 .

[5]  Clifford Nass,et al.  Does computer-generated speech manifest personality? an experimental test of similarity-attraction , 2000, CHI.

[6]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[7]  Tanzeem Choudhury,et al.  Automated Personalized Feedback for Physical Activity and Dietary Behavior Change With Mobile Phones: A Randomized Controlled Trial on Adults , 2015, JMIR mHealth and uHealth.

[8]  R. Calvo,et al.  Positive Computing: Technology for Wellbeing and Human Potential , 2014, Psychology Teaching Review.

[9]  Akane Sano,et al.  Predicting students' happiness from physiology, phone, mobility, and behavioral data , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[10]  R. A. Calvo,et al.  eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis , 2017, BMC Psychiatry.

[11]  Rosalind W. Picard,et al.  “Kind and Grateful”: A Context-Sensitive Smartphone App Utilizing Inspirational Content to Promote Gratitude , 2016, Psychology of well-being.

[12]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys.

[13]  Fabio Pianesi,et al.  Happiness Recognition from Mobile Phone Data , 2013, 2013 International Conference on Social Computing.

[14]  Rosalind W. Picard,et al.  BrightBeat: Effortlessly Influencing Breathing for Cultivating Calmness and Focus , 2017, CHI Extended Abstracts.

[15]  Steve Whittaker,et al.  Personality Depends on The Medium: Differences in Self-Perception on Snapchat, Facebook and Offline , 2018, CHI.

[16]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[17]  J. Russell.  A circumplex model of affect. , 1980 .

[18]  Orianna Demasi,et al.  Automated Text Messaging as an Adjunct to Cognitive Behavioral Therapy for Depression: A Clinical Trial , 2017, Journal of medical Internet research.

[19]  Kallirroi Georgila,et al.  SimSensei kiosk: a virtual human interviewer for healthcare decision support , 2014, AAMAS.

[20]  Alex Pentland,et al.  Using Social Sensing to Understand the Links between Sleep, Mood, and Sociability , 2011, 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing.

[21]  Sooyeon Jeong,et al.  Improving Smartphone Users' Affect and Wellbeing with Personalized Positive Psychology Interventions , 2016, HAI.

[22]  Andreas Paepcke,et al.  Conversational Agents and Mental Health: Theory-Informed Assessment of Language and Affect , 2016, HAI.

[23]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[24]  Akane Sano,et al.  HealthAware: An advice system for stress, sleep, diet and exercise , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[25]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[26]  Alex Pentland,et al.  Daily Stress Recognition from Mobile Phone Data, Weather Conditions and Individual Traits , 2014, ACM Multimedia.

[27]  Rosalind W. Picard,et al.  Text Messaging for Exercise Promotion in Older Adults From an Upper-Middle-Income Country: Randomized Controlled Trial , 2015, Journal of medical Internet research.

[28]  Gary Hsieh,et al.  Send Me a Different Message: Utilizing Cognitive Space to Create Engaging Message Triggers , 2017, CSCW.

[29]  Louis-Philippe Morency,et al.  It's only a computer: Virtual humans increase willingness to disclose , 2014, Comput. Hum. Behav..

[30]  Mirco Musolesi,et al.  My Phone and Me: Understanding People's Receptivity to Mobile Notifications , 2016, CHI.

[31]  Frederick Muench,et al.  Understanding Messaging Preferences to Inform Development of Mobile Goal-Directed Behavioral Interventions , 2014, Journal of medical Internet research.

[32]  Ning Wang,et al.  Creating Rapport with Virtual Agents , 2007, IVA.

[33]  Daniel McDuff,et al.  AffectAura: an intelligent system for emotional memory , 2012, CHI.

[34]  Rosalind W. Picard,et al.  Objective assessment of depressive symptoms with machine learning and wearable sensors data , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII).

[35]  Timothy Bickmore,et al.  An Affectively Aware Virtual Therapist for Depression Counseling , 2016 .

[36]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[37]  Arthur C. Graesser,et al.  Toward an Affect-Sensitive AutoTutor , 2007, IEEE Intelligent Systems.

[38]  Daniel McDuff,et al.  Towards Understanding Emotional Intelligence for Behavior Change Chatbots , 2019, 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).

[39]  Alex Pentland,et al.  Modeling the co-evolution of behaviors and social relationships using mobile phone data , 2011, MUM.

[40]  K. Fitzpatrick,et al.  Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial , 2017, JMIR mental health.

[41]  David C. Mohr,et al.  Ecological momentary interventions for depression and anxiety , 2017, Depression and anxiety.

[42]  James R McKay,et al.  A Randomized Controlled Pilot Trial of Different Mobile Messaging Interventions for Problem Drinking Compared to Weekly Drink Tracking , 2017, PloS one.

[43]  Yunxin Liu,et al.  Can Your Smartphone Infer Your Mood ? , 2011 .

[44]  R. Calvo,et al.  Application of Synchronous Text-Based Dialogue Systems in Mental Health Interventions: Systematic Review , 2017, Journal of medical Internet research.

[45]  Ran Gilad-Bachrach,et al.  PopTherapy: coping with stress through pop-culture , 2014, PervasiveHealth.

[46]  Paul Lukowicz,et al.  Can smartphones detect stress-related changes in the behaviour of individuals? , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops.

[47]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[48]  Akane Sano,et al.  Designing opportune stress intervention delivery timing using multi-modal data , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII).

[49]  D. Mohr,et al.  Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. , 2017, Annual review of clinical psychology.

[50]  Amit Baumel,et al.  More Than a Text Message: Dismantling Digital Triggers to Curate Behavior Change in Patient-Centered Health Interventions , 2017, Journal of medical Internet research.

[51]  P. Lovibond,et al.  The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. , 1995, Behaviour research and therapy.

[52]  Justine Cassell,et al.  Relational agents: a model and implementation of building user trust , 2001, CHI.