Multi-Sensor Based State Prediction for Personal Mobility Vehicles

This paper presents a study on multi-modal human emotional state detection while riding a powered wheelchair (PMV; Personal Mobility Vehicle) in an indoor labyrinth-like environment. The study reports findings on the habituation of human stress response during self-driving. In addition, the effects of “loss of controllability”, change in the role of the driver to a passenger, are investigated via an autonomous driving modality. The multi-modal emotional state detector sensing framework consists of four sensing devices: electroencephalograph (EEG), heart inter-beat interval (IBI), galvanic skin response (GSR) and stressor level lever (in the case of autonomous riding). Physiological emotional state measurement characteristics are organized by time-scale, in terms of capturing slower changes (long-term) and quicker changes from moment-to-moment. Experimental results with fifteen participants regarding subjective emotional state reports and commercial software measurements validated the proposed emotional state detector. Short-term GSR and heart signal characterizations captured moment-to-moment emotional state during autonomous riding (Spearman correlation; ρ = 0.6, p < 0.001). Short-term GSR and EEG characterizations reliably captured moment-to-moment emotional state during self-driving (Classification accuracy; 69.7). Finally, long-term GSR and heart characterizations were confirmed to reliably capture slow changes during autonomous riding and also of emotional state during participant resting state. The purpose of this study and the exploration of various algorithms and sensors in a structured framework is to provide a comprehensive background for multi-modal emotional state prediction experiments and/or applications. Additional discussion regarding the feasibility and utility of the possibilities of these concepts are given.

[1]  Li Bao,et al.  "Wheelchair slow transit" system-based elderly auxiliary travel mode , 2015 .

[2]  Milan Simic,et al.  In the Passenger Seat: Investigating Ride Comfort Measures in Autonomous Cars , 2015, IEEE Intelligent Transportation Systems Magazine.

[3]  Atsushi Watanabe,et al.  Including human factors for planning comfortable paths , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Tam Harbert,et al.  Japanese startup reinvents the wheelchair [News] , 2015 .

[5]  Wai Chong Chia,et al.  Analysis of Single-Electrode EEG Rhythms Using MATLAB to Elicit Correlation with Cognitive Stress , 2015 .

[6]  J. Polak,et al.  Autonomous cars: The tension between occupant experience and intersection capacity , 2015 .

[7]  Norihiro Hagita,et al.  Visibility analysis for autonomous vehicle comfortable navigation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Norihiro Hagita,et al.  Modeling of human velocity habituation for a robotic wheelchair , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Christian Büchel,et al.  Parametric trial-by-trial prediction of pain by easily available physiological measures , 2014, PAIN®.

[10]  Norihiro Hagita,et al.  Human-comfortable navigation for an autonomous robotic wheelchair , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Yang Wang,et al.  GSR and Blink Features for Cognitive Load Classification , 2013, INTERACT.

[12]  Ulrike Ehlert,et al.  The Effect of Music on the Human Stress Response , 2013, PloS one.

[13]  Joshua H. Balsters,et al.  Healthy aging is associated with increased neural processing of positive valence but attenuated processing of emotional arousal: an fMRI study , 2013, Neurobiology of Aging.

[14]  P. Melillo,et al.  Nonlinear Heart Rate Variability features for real-life stress detection. Case study: students under stress due to university examination , 2011, Biomedical engineering online.

[15]  Rachid Alami,et al.  Physiological and subjective evaluation of a human-robot object hand-over task. , 2011, Applied ergonomics.

[16]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[17]  Ian Pitt,et al.  Evaluating a Brain-Computer Interface to Categorise Human Emotional Response , 2010, 2010 10th IEEE International Conference on Advanced Learning Technologies.

[18]  M. Benedek,et al.  A continuous measure of phasic electrodermal activity , 2010, Journal of Neuroscience Methods.

[19]  V. Pavlenko,et al.  EEG Correlates of Anxiety and Emotional Stability in Adult Healthy Subjects , 2009, Neurophysiology.

[20]  Richard F. Thompson Habituation: A history , 2009, Neurobiology of Learning and Memory.

[21]  Donald A. Wilson,et al.  Habituation revisited: An updated and revised description of the behavioral characteristics of habituation , 2009, Neurobiology of Learning and Memory.

[22]  E. Verona,et al.  Stress-induced asymmetric frontal brain activity and aggression risk. , 2009, Journal of abnormal psychology.

[23]  Benjamin Kuipers,et al.  High performance control for graceful motion of an intelligent wheelchair , 2008, 2008 IEEE International Conference on Robotics and Automation.

[24]  Nicole Y. Weekes,et al.  The effect of a naturalistic stressor on frontal EEG asymmetry, stress, and health , 2007, Biological Psychology.

[25]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[26]  L Fehr,et al.  Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. , 2000, Journal of rehabilitation research and development.

[27]  H. Critchley,et al.  Neural Activity Relating to Generation and Representation of Galvanic Skin Conductance Responses: A Functional Magnetic Resonance Imaging Study , 2000, The Journal of Neuroscience.

[28]  L. F. Barrett Discrete Emotions or Dimensions? The Role of Valence Focus and Arousal Focus , 1998 .

[29]  E. Basar,et al.  Alpha oscillations in brain functioning: an integrative theory. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  L. Feldman Valence Focus and Arousal Focus: Individual Differences in the Structure of Affective Experience , 1995 .

[31]  Shin'ichi Yuta,et al.  Vehicle command system and trajectory control for autonomous mobile robots , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[32]  J. Russell,et al.  A cross-cultural study of a circumplex model of affect. , 1989 .

[33]  S. Snodgrass The Effects of Walking Behavior on Mood. , 1986 .

[34]  P. Ekman,et al.  Autonomic nervous system activity distinguishes among emotions. , 1983, Science.

[35]  Norman R. Draper,et al.  Applied regression analysis (2. ed.) , 1981, Wiley series in probability and mathematical statistics.

[36]  H Zeier,et al.  Concurrent physiological activity of driver and passenger when driving with and without automatic transmission in heavy city traffic. , 1979, Ergonomics.

[37]  F. Braceland THE STRESS OF LIFE , 1976 .

[38]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[39]  Jessica Hendricks,et al.  Alternative Wheelchair Control System , 2013 .

[40]  Marina Rodríguez,et al.  Mental Stress Detection Using Multimodal Sensing in a Wireless Body Area Network , 2012, Informatiktage.

[41]  Neerincx,et al.  EEG alpha asymmetry, heart rate variability and cortisol in response to virtual reality induced stress , 2011 .

[42]  H. Storma,et al.  The development of a software program for analyzing spontaneous and externally elicited skin conductance changes in infants and adults , 2000 .

[43]  C. Spielberger,et al.  Manual for the State-Trait Anxiety Inventory , 1970 .

[44]  D. M. Ellis,et al.  Applied Regression Analysis , 1968 .

[45]  R. F. Thompson,et al.  Habituation: a model phenomenon for the study of neuronal substrates of behavior. , 1966, Psychological review.