Ambulatory Assessment of Affect: Survey of Sensor Systems for Monitoring of Autonomic Nervous Systems Activation in Emotion

Advances in miniaturized computing, storage and communication resources for personal wearable electronics devices, as well as the availability of diverse sensors for state assessment enable the development of a wide variety of wearable body area network (BAN) systems for psychophysiological measurements. These systems pave the way for acquisition of quality data relevant for research studies, amongst others, on the autonomic nervous system (ANS) activation in emotion. We present a high-level overview of BAN and its features, and we review 173 publications that report research studies on emotion activation and particularly 15 BAN systems employed in these studies. We discuss each BAN in terms of its capacity for ambulatory, i.e., out of the laboratory, assessment of the ANS activation in emotion. Finally, we highlight the design challenges to be addressed to make BAN systems effective for a wide range of applications to support users wellbeing and overall Quality of Life improvement. This paper provides knowledge to those interested in (ambulatory) assessment of the ANS activation on the set of systems currently used in research, and it aims to highlight opportunities for scientists and practitioners in, amongst others, the affective computing domain, enabling them to reflect upon their BAN requirements and study designs.

[1]  T. Trull,et al.  Ambulatory Assessment : An Innovative and Promising Approach for Clinical Psychology , 2009 .

[2]  Jennifer Healey,et al.  Recording Affect in the Field: Towards Methods and Metrics for Improving Ground Truth Labels , 2011, ACII.

[3]  M. Maclachlan,et al.  Naturalistic monitoring of the affect-heart rate relationship: a day reconstruction study. , 2010, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[4]  G. Alpers,et al.  Psychophysiological assessment during exposure in driving phobic patients. , 2005, Journal of abnormal psychology.

[5]  D. Watson,et al.  Mood, blood pressure, and heart rate at work: an experience-sampling study. , 2010, Journal of occupational health psychology.

[6]  E. D. de Geus,et al.  Overcommitment to Work Is Associated With Changes in Cardiac Sympathetic Regulation , 2004, Psychosomatic medicine.

[7]  B. Ditto,et al.  Asthma self-efficacy, high frequency heart rate variability, and airflow obstruction during negative affect in daily life. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[8]  G. Alpers,et al.  And yet they correlate: psychophysiological activation predicts self-report outcomes of exposure therapy in claustrophobia. , 2008, Journal of anxiety disorders.

[9]  Rosalind W. Picard,et al.  A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity , 2010, IEEE Transactions on Biomedical Engineering.

[10]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[11]  J. Fahrenberg,et al.  Ambulatory assessment : computer-assisted psychological and psychophysiological methods in monitoring and field studies , 1999 .

[12]  T. Kamarck,et al.  Effects of task strain, social conflict, and emotional activation on ambulatory cardiovascular activity: daily life consequences of recurring stress in a multiethnic adult sample. , 1998, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[13]  Frank H. Wilhelm,et al.  Continuous electronic data capture of physiology, behavior and experience in real life: towards ecological momentary assessment of emotion , 2006, Interact. Comput..

[14]  K. Conrad,et al.  Effects of meals on hemodynamics: Implications for antihypertensive drug studies , 1986, Clinical pharmacology and therapeutics.

[15]  Schaefer,et al.  Psychological response patterning in emotion: implications for affective computing , 2010 .

[16]  K. Matthews,et al.  Effects of hostility on ambulatory blood pressure and mood during daily living in healthy adults. , 1999, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[17]  W. Roth,et al.  End-Tidal pCO2 in Blood Phobics During Viewing of Emotion- and Disease-Related Films , 2005, Psychosomatic medicine.

[18]  M. A. Rowe,et al.  Guide for Analysing Electrodermal Activity & Skin Conductance Responses for Psychological Experiments , 2013 .

[19]  W. Roth,et al.  Respiratory Biofeedback-Assisted Therapy in Panic Disorder , 2001, Behavior modification.

[20]  Maria Blettner,et al.  Sample size calculation in clinical trials: part 13 of a series on evaluation of scientific publications. , 2010, Deutsches Arzteblatt international.

[21]  R M Williams-Avery,et al.  Cardiovascular changes during induced emotion: an application of lang's theory of emotional imagery. , 1999, Journal of psychosomatic research.

[22]  R. Freedman,et al.  Ambulatory monitoring of panic disorder. , 1985, Archives of general psychiatry.

[23]  J. Olesen,et al.  The economic cost of brain disorders in Europe , 2012, European journal of neurology.

[24]  Comparison of hemodynamic responses to social and nonsocial stress: evaluation of an anger interview. , 2001, Psychophysiology.

[25]  Wan Li,et al.  Why do you sigh? Sigh rate during induced stress and relief. , 2009, Psychophysiology.

[26]  J. Gehricke,et al.  Effects of transdermal nicotine on symptoms, moods, and cardiovascular activity in the everyday lives of smokers and nonsmokers with attention-deficit/hyperactivity disorder. , 2009, Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors.

[27]  J. Andreassi.  Psychophysiology: Human Behavior and Physiological Response , 1980 .

[28]  R. Carels,et al.  The association between emotional upset and cardiac arrhythmia during daily life. , 2003, Journal of consulting and clinical psychology.

[29]  J. Swanson,et al.  Reducing Hyperactivity with a Feedback Actigraph: Initial Findings , 2006, Clinical child psychology and psychiatry.

[30]  Renate Rau,et al.  Psychophysiological assessment of human reliability in a simulated complex system , 1996, Biological Psychology.

[31]  Shein-Chung Chow,et al.  Design and Analysis of Clinical Trials: Concepts and Methodologies , 2013 .

[32]  ChittaroLuca,et al.  Affective computing vs. affective placebo , 2014 .

[33]  Gary G Berntson,et al.  Loneliness in everyday life: cardiovascular activity, psychosocial context, and health behaviors. , 2003, Journal of personality and social psychology.

[34]  Julian F. Thayer,et al.  Cardiac Effects of Momentary Assessed Worry Episodes and Stressful Events , 2007, Psychosomatic medicine.

[35]  G. Miller,et al.  Science Perspectives on Psychological the Smartphone Psychology Manifesto on Behalf Of: Association for Psychological Science the Smartphone Psychology Manifesto Previous Research Using Mobile Electronic Devices What Smartphones Can Do Now and Will Be Able to Do in the near Future , 2022 .

[36]  P. Stone,et al.  The clinical significance of continuous ECG (ambulatory ECG or Holter) monitoring of the ST-segment to evaluate ischemia: a review. , 2013, Progress in cardiovascular diseases.

[37]  F. Wilhelm,et al.  Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment , 2010, Biological Psychology.

[38]  Marcus A. Krupp,et al.  Current medical diagnosis & treatment , 1974 .

[39]  P. Delespaul,et al.  Translating assessments of the film of daily life into person‐tailored feedback interventions in depression , 2011, Acta psychiatrica Scandinavica.

[40]  Georg Brügner,et al.  ECG Changes, Emotional Arousal, and Subjective State , 2000 .

[41]  T. Kamarck,et al.  Hostility moderates the effects of social support and intimacy on blood pressure in daily social interactions. , 2008, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[42]  M. Bakermans-Kranenburg,et al.  Parent-child relationship, temperament, and physiological reactions to fear-inducing film clips: further evidence for differential susceptibility. , 2008, Journal of experimental child psychology.

[43]  T. Conner,et al.  Getting started: Launching a study in daily life. , 2012 .

[44]  J. Os,et al.  Cost of disorders of the brain in Europe 2010 , 2011, European Neuropsychopharmacology.

[45]  J. Olesen,et al.  Cost of disorders of the brain in Europe , 2005, European journal of neurology.

[46]  Thomas Kubiak,et al.  Psychological and Psychophysiological Ambulatory Monitoring A Review of Hardware and Software Solutions , 2007 .

[47]  S. Filipović,et al.  Psychophysiology: Human Behavior and Physiological Response (4th ed.) , 2001 .

[48]  Hazem M. Hajj,et al.  A survey of ground-truth in emotion data annotation , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops.

[49]  T. Kamarck,et al.  Partner Interactions Are Associated With Reduced Blood Pressure in the Natural Environment: Ambulatory Monitoring Evidence From a Healthy, Multiethnic Adult Sample , 2001, Psychosomatic medicine.

[50]  Sheri L. Johnson,et al.  Risk for mania and positive emotional responding: too much of a good thing? , 2008, Emotion.

[51]  D. Kahneman,et al.  A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method , 2004, Science.

[52]  Frank H Wilhelm,et al.  Physical activity and respiratory behavior in daily life of patients with panic disorder and healthy controls. , 2010, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[53]  Jan H. Houtveen,et al.  Circadian variation in base rate measures of cardiac autonomic activity , 2004, European Journal of Applied Physiology.

[54]  K. Pawlik,et al.  Ambulatory Assessment - Monitoring Behavior in Daily Life Settings A Behavioral-Scientific Challenge for Psychology , 2007 .

[55]  L. V. van Doornen,et al.  Mobile Web-Based Monitoring and Coaching: Feasibility in Chronic Migraine , 2007, Journal of medical Internet research.

[56]  Tushar Sakpal,et al.  Sample Size Estimation in Clinical Trial , 2010, Perspectives in clinical research.

[57]  Isabelle Peretz,et al.  Role of tempo entrainment in psychophysiological differentiation of happy and sad music? , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[58]  P. Grossman,et al.  Patterns of Objective Physical Functioning and Perception of Mood and Fatigue in Posttreatment Breast Cancer Patients and Healthy Controls: An Ambulatory Psychophysiological Investigation , 2008, Psychosomatic medicine.

[59]  Ulrich Ebner-Priemer,et al.  Ambulatory assessment. , 2013, Annual review of clinical psychology.

[60]  B. Dahme,et al.  Airway response to emotional stimuli in asthma: the role of the cholinergic pathway. , 2010, Journal of applied physiology.

[61]  E. J. Geus,et al.  Noninvasive Psychophysiological Ambulatory Recordings : Study Design and Data Analysis Strategies , 2009 .

[62]  M. Mehl,et al.  Handbook of research methods for studying daily life , 2012 .

[63]  B. Fredrickson.  The role of positive emotions in positive psychology. The broaden-and-build theory of positive emotions. , 2001, The American psychologist.

[64]  D. Volterrani,et al.  Assessment of physical and mental activity using a standardized computer‐assisted diary during ambulatory blood pressure monitoring , 1993, Journal of hypertension. Supplement : official journal of the International Society of Hypertension.

[65]  Janice R. Kuo,et al.  Distress and Affective Dysregulation in Patients With Borderline Personality Disorder: A Psychophysiological Ambulatory Monitoring Study , 2008, The Journal of nervous and mental disease.

[66]  Philippe Georges Zimmermann,et al.  Valence lasts longer than arousal : persistence of induced moods as assessed by psychophysiological measures , 2009 .

[67]  T. Meijman,et al.  Vagal cardiac control throughout the day: the relative importance of effort–reward imbalance and within-day measurements of mood, demand and satisfaction , 2001, Biological Psychology.

[68]  Giuseppe Mancia,et al.  EFFECTS OF BLOOD-PRESSURE MEASUREMENT BY THE DOCTOR ON PATIENT'S BLOOD PRESSURE AND HEART RATE , 1983, The Lancet.

[69]  M. Chung.  The role of the wearable cardioverter defibrillator in clinical practice. , 2014, Cardiology clinics.

[70]  S. Shiffman,et al.  20. Ecological Momentary Assessment , 2013 .

[71]  Lui Sha,et al.  Modeling and architecture design of an MDPnP acute care monitoring system , 2013, Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems.

[72]  Friedrich Foerster,et al.  Motion pattern and posture: Correctly assessed by calibrated accelerometers , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[73]  Ryuji Kohno,et al.  Measurement of Core Body Temperature by an Ingestible Capsule Sensor and Evaluation of its Wireless Communication Performance , 2012 .

[74]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[75]  W Linden,et al.  A microanalysis of autonomic activity during human speech. , 1987, Psychosomatic medicine.

[76]  D. Bassett,et al.  The technology of accelerometry-based activity monitors: current and future. , 2005, Medicine and science in sports and exercise.

[77]  Thomas Prill,et al.  New methods in ambulatory blood pressure monitoring: Interactive monitoring and detection of posture and movement patterns , 2007, Behavior research methods.

[78]  M. De Hert,et al.  Cost of disorders of the brain in Europe. , 2006, European journal of neurology.

[79]  P. Ekman.  An argument for basic emotions , 1992 .

[80]  P. Lang.  International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[81]  Selim Ickin,et al.  Factors influencing quality of experience of commonly used mobile applications , 2012, IEEE Communications Magazine.

[82]  James P. Smith,et al.  Long-term economic costs of psychological problems during childhood. , 2010, Social science & medicine.

[83]  K. Wac,et al.  Mobile patient monitoring: The MobiHealth system , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[84]  C. Gehin,et al.  EmoSense: An Ambulatory Device for the Assessment of ANS Activity—Application in the Objective Evaluation of Stress With the Blind , 2012, IEEE Sensors Journal.

[85]  J. Smyth,et al.  Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. , 2010, British journal of health psychology.

[86]  Alan C. Swann,et al.  The Lifetime Cost of Bipolar Disorder in the US , 2012, PharmacoEconomics.

[87]  T. Kamarck,et al.  The Diary of Ambulatory Behavioral States: A new approach to the assessment of psychosocial influences on ambulatory cardiovascular activity. , 1998 .

[88]  Matthew F. Muldoon,et al.  Psychosocial demands and ambulatory blood pressure: a field assessment approach , 2002, Physiology & Behavior.

[89]  H. Bussmann,et al.  Ambulatory monitoring of mobility-related activities in rehabilitation medicine , 1998 .

[90]  D. Dijk,et al.  Circadian and sleep/wake dependent aspects of subjective alertness and cognitive performance , 1992, Journal of sleep research.

[91]  A. Schwerdtfeger,et al.  Social interaction moderates the relationship between depressive mood and heart rate variability: evidence from an ambulatory monitoring study. , 2009, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[92]  Daniel Arthur James,et al.  Sensors and sensor systems for psychophysiological monitoring: A review of current trends , 2007 .

[93]  S Folkard,et al.  Complex interaction of the sleep-wake cycle and circadian phase modulates mood in healthy subjects. , 1997, Archives of general psychiatry.

[94]  W. Roth,et al.  Distinguishing emotional from physical activation in ambulatory psychophysiological monitoring. , 2006, Biomedical sciences instrumentation.

[95]  Oscar Mayora-Ibarra,et al.  Using smart phone mobility traces for the diagnosis of depressive and manic episodes in bipolar patients , 2014, AH.

[96]  S. Shiffman,et al.  Ecological momentary assessment. , 2008, Annual review of clinical psychology.

[97]  Luca Chittaro,et al.  Affective computing vs. affective placebo: Study of a biofeedback-controlled game for relaxation training , 2014, Int. J. Hum. Comput. Stud..

[98]  I. Christie,et al.  Autonomic specificity of discrete emotion and dimensions of affective space: a multivariate approach. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[99]  E. D. de Geus,et al.  Effects of work stress on ambulatory blood pressure, heart rate, and heart rate variability. , 2000, Hypertension.

[100]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[101]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[102]  C. Adair,et al.  E-mental health: a rapid review of the literature. , 2014, Psychiatric services.

[103]  F. Wilhelm,et al.  Respiratory sinus arrhythmia, cardiac vagal control, and daily activity. , 2004, American journal of physiology. Heart and circulatory physiology.

[104]  M. Massagli,et al.  Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm , 2011, Nature Biotechnology.

[105]  Stefan Hey,et al.  The Association between Short Periods of Everyday Life Activities and Affective States: A Replication Study Using Ambulatory Assessment , 2013, Front. Psychol..

[106]  Timothy W. Smith,et al.  Age-related differences in ambulatory blood pressure during daily stress: evidence for greater blood pressure reactivity with age. , 2006, Psychology and aging.

[107]  Vasant Dhar,et al.  Data science and prediction , 2012, CACM.

[108]  Victor C. M. Leung,et al.  Enabling technologies for wireless body area networks: A survey and outlook , 2009, IEEE Communications Magazine.

[109]  Athanasios V. Vasilakos,et al.  Body area networks for ambulatory psychophysiological monitoring: a survey of off-the-shelf sensor systems , 2010, BODYNETS.

[110]  Jennifer Healey,et al.  Out of the Lab and into the Fray: Towards Modeling Emotion in Everyday Life , 2010, Pervasive.

[111]  U. Ebner-Priemer,et al.  Psychophysiological ambulatory assessment of affective dysregulation in borderline personality disorder , 2007, Psychiatry Research.