Signal Processing of Multimodal Mobile Lifelogging Data Towards Detecting Stress in Real-World Driving

Stress is a negative emotion that is part of everyday life. However, frequent episodes or prolonged periods of stress can be detrimental to long-term health. Nevertheless, developing self-awareness is an important aspect of fostering effective ways to self-regulate these experiences. Mobile lifelogging systems provide an ideal platform to support self-regulation of stress by raising awareness of negative emotional states via continuous recording of psychophysiological and behavioral data. However, obtaining meaningful information from large volumes of raw data represents a significant challenge because these data must be accurately quantified and processed before stress can be detected. This work describes a set of algorithms designed to process multiple streams of lifelogging data for stress detection in the context of real world driving. Two data collection exercises have been performed where multimodal data, including raw cardiovascular activity and driving information, were collected from 21 people during daily commuter journeys. Our approach enabled us to 1) pre-process raw physiological data to calculate valid measures of heart rate variability, a significant marker of stress, 2) identify/correct artefacts in the raw physiological data, and 3) provide a comparison between several classifiers for detecting stress. Results were positive and ensemble classification models provided a maximum accuracy of 86.9 percent for binary detection of stress in the real-world.

[1]  J. Dimsdale,et al.  The relationship between heart rate variability and inflammatory markers in cardiovascular diseases , 2008, Psychoneuroendocrinology.

[2]  Andrew J. May,et al.  A Comparison of Field-Based and Lab-Based Experiments to Evaluate User Experience of Personalised Mobile Devices , 2013, Adv. Hum. Comput. Interact..

[3]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[4]  Marko Robnik-Sikonja,et al.  Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF , 2004, Applied Intelligence.

[5]  NovakDomen,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012 .

[6]  Jerry Suls,et al.  Anger, anxiety, and depression as risk factors for cardiovascular disease: the problems and implications of overlapping affective dispositions. , 2005, Psychological bulletin.

[7]  Ronald Glaser,et al.  Emotions, morbidity, and mortality: new perspectives from psychoneuroimmunology. , 2002, Annual review of psychology.

[8]  Anita L. Allen,et al.  Dredging Up the Past: Lifelogging, Memory and Surveillance , 2007 .

[9]  Hsien-Tsai Wu,et al.  Measuring Pulse Wave Velocity Using ECG and Photoplethysmography , 2011, Journal of Medical Systems.

[10]  Daniel McDuff,et al.  AffectAura: an intelligent system for emotional memory , 2012, CHI.

[11]  Mary Czerwinski,et al.  MoodWings: a wearable biofeedback device for real-time stress intervention , 2013, PETRA '13.

[12]  R. Magno,et al.  Coronary heart disease , 1957 .

[13]  Chelsea Dobbins,et al.  Lesson Learned from Collecting Quantified Self Information via Mobile and Wearable Devices , 2015, J. Sens. Actuator Networks.

[14]  Abigail Sellen,et al.  Do life-logging technologies support memory for the past?: an experimental study using sensecam , 2007, CHI.

[15]  Amine Naït-Ali,et al.  QRS complex detection using Empirical Mode Decomposition , 2010, Digit. Signal Process..

[16]  Jari Viik,et al.  Perceived Mental Stress and Reactions in Heart Rate Variability—A Pilot Study Among Employees of an Electronics Company , 2008, International journal of occupational safety and ergonomics : JOSE.

[17]  Springer-Verlag London Limited UbiqLog: a generic mobile phone-based life-log framework , 2012 .

[18]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[19]  V Kalidas,et al.  Cardiac arrhythmia classification using multi-modal signal analysis , 2016, Physiological measurement.

[20]  Dylan M. Jones,et al.  Refining the measurement of mood: The UWIST Mood Adjective Checklist , 1990 .

[21]  J. Thayer,et al.  A careful look at ECG sampling frequency and R-peak interpolation on short-term measures of heart rate variability , 2015, Physiological measurement.

[22]  Myounghoon Jeon,et al.  Detection of Drivers' Incidental and Integral Affect Using Physiological Measures , 2013 .

[23]  Christos D. Katsis,et al.  Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[24]  W. C. Elliott,et al.  Coronary Heart Disease , 2002, Encyclopedia of Behavioral Medicine.

[25]  Chelsea Dobbins,et al.  Detecting Negative Emotions During Real-Life Driving via Dynamically Labelled Physiological Data , 2018, 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[26]  Richard P. Sloan,et al.  Heart rate variability predicts levels of inflammatory markers: Evidence for the vagal anti-inflammatory pathway , 2015, Brain, Behavior, and Immunity.

[27]  Giuseppe De Pietro,et al.  Indirect Blood Pressure Evaluation by Means of Genetic Programming , 2015, BIOSTEC.

[28]  Rahul Banerjee,et al.  Assessment of Driver Stress from Physiological Signals collected under Real-Time Semi-Urban Driving Scenarios , 2014, Int. J. Comput. Intell. Syst..

[29]  Alan F. Smeaton,et al.  Experiences of Aiding Autobiographical Memory Using the SenseCam , 2012, Hum. Comput. Interact..

[30]  G. Lu,et al.  A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects , 2009, Journal of medical engineering & technology.

[31]  Shahram Izadi,et al.  SenseCam: A Retrospective Memory Aid , 2006, UbiComp.

[32]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[33]  Mobyen Uddin Ahmed,et al.  K-NN based interpolation to handle artifacts for heart rate variability analysis , 2011, 2011 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[34]  R. D. Grainger,et al.  Managing stress. , 2010, The American journal of nursing.

[35]  Michele Risi,et al.  Visual Mobile Computing for Mobile End-Users , 2016, IEEE Transactions on Mobile Computing.

[36]  Matthias Rauterberg,et al.  Unconscious emotions: quantifying and logging something we are not aware of , 2013, Personal and Ubiquitous Computing.

[37]  Kai Li,et al.  Study of selective ensemble learning method and its diversity based on decision tree and neural network , 2010, 2010 Chinese Control and Decision Conference.

[38]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[39]  Moushumi Sharmin,et al.  Estimating Drivers' Stress from GPS Traces , 2014, AutomotiveUI.

[40]  Baozhi Chen,et al.  Research challenges in computation, communication, and context awareness for ubiquitous healthcare , 2012, IEEE Communications Magazine.

[41]  J. S. Ubhi,et al.  Comparative study of FIR and IIR filters for the removal of Baseline noises from ECG signal , 2011 .

[42]  Akane Sano,et al.  Predicting students' happiness from physiology, phone, mobility, and behavioral data , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[43]  Emre Ertin,et al.  cStress: towards a gold standard for continuous stress assessment in the mobile environment , 2015, UbiComp.

[44]  Gregory D. Abowd,et al.  Getting into the Living Memory Box: Family archives & holistic design , 2003, Personal and Ubiquitous Computing.

[45]  N. Maglaveras,et al.  Association of heart rate variability and inflammatory response in patients with cardiovascular diseases: current strengths and limitations , 2013, Front. Physiol..

[46]  Kejia Li,et al.  Initial study on pulse wave velocity acquired from one hand using two synchronized wireless reflectance pulse oximeters , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[47]  Mobyen Uddin Ahmed,et al.  Heart Rate and Inter-beat Interval Computation to Diagnose Stress Using ECG Sensor Signal , 2010 .