Smartphone-Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods

The identification of daily life events that trigger significant changes on our affective state has become a fundamental task in emotional research. To achieve it, the affective states must be assessed in real-time, along with situational information that could contextualize the affective data acquired. However, the objective monitoring of the affective states and the context is still in an early stage. Mobile technologies can help to achieve this task providing immediate and objective data of the users’ context and facilitating the assessment of their affective states. Previous works have developed mobile apps for monitoring affective states and context, but they use a fixed methodology which does not allow for making changes based on the progress of the study. This work presents a multimodal platform which leverages the potential of the smartphone sensors and the Experience Sampling Methods (ESM) to provide a continuous monitoring of the affective states and the context in an ubiquitous way. The platform integrates several elements aimed to expedite the real-time management of the ESM questionnaires. In order to show the potential of the platform, and evaluate its usability and its suitability for real-time assessment of affective states, a pilot study has been conducted. The results demonstrate an excellent usability level and a good acceptance from the users and the specialists that conducted the study, and lead to some suggestions for improving the data quality of mobile context-aware ESM-based systems.

[1]  H. Riper,et al.  Mobile Phone-Based Unobtrusive Ecological Momentary Assessment of Day-to-Day Mood: An Explorative Study , 2016, Journal of medical Internet research.

[2]  U. Schimmack Affect Measurement in Experience Sampling Research , 2003 .

[3]  R. Kessler,et al.  Measuring daily events and experiences: decisions for the researcher. , 1991, Journal of personality.

[4]  Bert Arnrich,et al.  Utilizing Emoticons on Mobile Devices within ESM studies to Measure Emotions in the Field , 2009 .

[5]  Jorge Gonçalves,et al.  Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports , 2019, Int. J. Hum. Comput. Stud..

[6]  J van Os,et al.  Use of the experience sampling method in the context of clinical trials , 2016, Evidence-Based Mental Health.

[7]  Ulrich W. Ebner-Priemer,et al.  How to Investigate Within-Subject Associations between Physical Activity and Momentary Affective States in Everyday Life: A Position Statement Based on a Literature Overview , 2013, Front. Psychol..

[8]  Mona Choi,et al.  Methodological Strategies for Ecological Momentary Assessment to Evaluate Mood and Stress in Adult Patients Using Mobile Phones: Systematic Review , 2019, JMIR mHealth and uHealth.

[9]  Robert Ferrer,et al.  Dynamic patterns in mood among newly diagnosed patients with major depressive episode or panic disorder and normal controls. , 2007, Primary care companion to the Journal of clinical psychiatry.

[10]  P. Ekkekakis A f f e c t , M o o d , a n d E m o t i o n , 2022 .

[11]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[12]  Hermie Hermens,et al.  MobileCogniTracker - A mobile experience sampling tool for tracking cognitive behaviour , 2019, J. Ambient Intell. Humaniz. Comput..

[13]  B. Parkinson Emotions are social. , 1996, British journal of psychology.

[14]  P. Ekman Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life , 2003 .

[15]  Marika Wenemark,et al.  Applying Motivation Theory to Achieve Increased Respondent Satisfaction, Response Rate and Data Quality in a Self-administered Survey , 2011 .

[16]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[17]  A. Persson,et al.  Applying Motivation Theory to Achieve Increased Response Rates , Respondent Satisfaction and Data Quality , 2011 .

[18]  Michael Eid,et al.  Psychometric properties of a computerized mobile phone method for assessing mood in daily life. , 2010, Emotion.

[19]  M. Csíkszentmihályi,et al.  Validity and Reliability of the Experience‐Sampling Method , 1987, The Journal of nervous and mental disease.

[20]  Niels van Berkel,et al.  The Experience Sampling Method on Mobile Devices , 2017, ACM Comput. Surv..

[21]  Virginie Nicaise,et al.  Emotion profiles and their motivational antecedents among adolescent athletes in intensive training settings , 2018 .

[22]  D. Watson,et al.  Toward a consensual structure of mood. , 1985, Psychological bulletin.

[23]  Denzil Ferreira,et al.  AWARE: Mobile Context Instrumentation Framework , 2015, Front. ICT.

[24]  W. Fleeson,et al.  A guide for data cleaning in experience sampling studies. , 2012 .

[25]  F. Naughton,et al.  Response Parameters for SMS Text Message Assessments Among Pregnant and General Smokers Participating in SMS Cessation Trials , 2015, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[26]  Choong Seon Hong,et al.  Human Behavior Analysis by Means of Multimodal Context Mining , 2016, Sensors.

[27]  P. Ekman,et al.  The nature of emotion: Fundamental questions. , 1994 .

[28]  D. Mohr,et al.  Harnessing Context Sensing to Develop a Mobile Intervention for Depression , 2011, Journal of medical Internet research.

[29]  Jorge Gonçalves,et al.  Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports , 2019, CHI.

[30]  Anind K. Dey,et al.  Understanding and Using Context , 2001, Personal and Ubiquitous Computing.

[31]  Guodong Sun,et al.  Daily Mood Assessment Based on Mobile Phone Sensing , 2012, 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks.

[32]  Héctor Pomares,et al.  Intelligent Monitoring of Affective Factors Underlying Sport Performance by Means of Wearable and Mobile Technology , 2018, UCAmI.

[33]  Richard J. Duro,et al.  Ambient Intelligence Systems for Personalized Sport Training , 2010, Sensors.

[34]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[35]  Hunter Gehlbach,et al.  Anchoring and Adjusting in Questionnaire Responses , 2012 .

[36]  J. Russell A circumplex model of affect. , 1980 .

[37]  K. Naragon-Gainey Affective models of depression and anxiety: Extension to within-person processes in daily life. , 2019, Journal of affective disorders.

[38]  Hoang Huu Son Toward a Computational Model of Mood , 2017, FNC/MobiSPC.

[39]  J. Smyth,et al.  Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. , 2010, British journal of health psychology.

[40]  Gabriella Kazai,et al.  Towards a science of user engagement. , 2011 .

[41]  Robert Li Kam Wa MoodScope: Building a Mood Sensor from Smartphone Usage Patterns , 2012 .

[42]  N. Bolger,et al.  Diary methods: capturing life as it is lived. , 2003, Annual review of psychology.

[43]  Oscar Mayora-Ibarra,et al.  Mobile phones as medical devices in mental disorder treatment: an overview , 2014, Personal and Ubiquitous Computing.

[44]  D. Mohr,et al.  Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. , 2017, Annual review of clinical psychology.

[45]  Ming-Hui Huang,et al.  Designing website attributes to induce experiential encounters , 2003, Comput. Hum. Behav..

[46]  G. J,et al.  Nonlinear dynamics of mood regulation in unaffected first-degree relatives of bipolar disorder patients. , 2019, Journal of affective disorders.

[47]  M. Wooden,et al.  Re‐engaging with survey non‐respondents: evidence from three household panels , 2014 .

[48]  N. Rickard,et al.  Development of a Mobile Phone App to Support Self-Monitoring of Emotional Well-Being: A Mental Health Digital Innovation , 2016, JMIR mental health.

[49]  Yunxin Liu,et al.  MoodScope: building a mood sensor from smartphone usage patterns , 2013, MobiSys.

[50]  Wolfgang Schlicht,et al.  Be active and become happy: an ecological momentary assessment of physical activity and mood. , 2010, Journal of sport & exercise psychology.