Classification of the Emotional State of a Subject Using Machine Learning Algorithms for RehabRoby

Robot-assisted rehabilitation systems have shown to be helpful in neuromotor rehabilitation because it is possible to deliver interactive and repeatable sensorimotor exercise and monitor the actual performance continuously. Note that it is also essential to distinguish if subject finds the rehabilitation task difficult or easy, since the difficulty level of a task can yield different emotional state, such as excited, bored, over-stressed, etc., at each subject. It is important to adjust the difficulty level of the task to encourage the non-motivated subjects during the therapy. The physiological measurements, which can be obtained from the biofeedback sensors, can be used to estimate the subject’s emotional state during the execution of the rehabilitation task. Machine learning methods can be used to classify the emotional state using the features of the biofeedback sensory data. This is explored in this chapter.

[1]  William S. Harwin,et al.  Upper Limb Robot Mediated Stroke Therapy—GENTLE/s Approach , 2003, Auton. Robots.

[2]  Marko Munih,et al.  A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing , 2012, Interact. Comput..

[3]  Michael Alexander,et al.  Passive exoskeletons for assisting limb movement. , 2006, Journal of rehabilitation research and development.

[4]  Ana L. N. Fred,et al.  Clustering Data with Temporal Evolution: Application to Electrophysiological Signals , 2010, ICAART.

[5]  Cristina Conati,et al.  Using Feature Selection and Unsupervised Clustering to Identify Affective Expressions in Educational Games , 2016 .

[6]  Makoto Sasaki,et al.  Development of a 3DOF mobile exoskeleton robot for human upper-limb motion assist , 2008, Robotics Auton. Syst..

[7]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Daniela M. Romano,et al.  Bio-Affective Computer Interface for Game Interaction , 2010, Int. J. Gaming Comput. Mediat. Simulations.

[9]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[10]  Maki K. Habib Biomimetics: innovations and robotics , 2011 .

[11]  P. Pound,et al.  A critical review of the concept of patient motivation in the literature on physical rehabilitation. , 2000, Social science & medicine.

[12]  Nilanjan Sarkar,et al.  Anxiety-based affective communication for implicit human–machine interaction , 2022 .

[13]  D. Brodie,et al.  Effects of Short-Term Psychological Stress on the Time and Frequency Domains of Heart-Rate Variability , 2000, Perceptual and motor skills.

[14]  J. Patton,et al.  Functional Restoration for the Stroke Survivor: Informing the Efforts of Engineers , 2008, Topics in stroke rehabilitation.

[15]  M.K. Habib,et al.  An emotion-based task sharing approach for a cooperative multiagent robotic system , 2008, 2008 IEEE International Conference on Mechatronics and Automation.

[16]  Duygun Erol,et al.  Design and Control of an Upper Limb Exoskeleton Robot RehabRoby , 2011, TAROS.

[17]  Maarouf Saad,et al.  Modeling and control of a 7DOF exoskeleton robot for arm movements , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[18]  R. Riener,et al.  Real-Time Closed-Loop Control of Cognitive Load in Neurological Patients During Robot-Assisted Gait Training , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Keigo Watanabe,et al.  Affection Based Multi-robot Team Work , 2008 .

[20]  Melih Kandemir,et al.  Learning Mental States from Biosignals , 2013 .

[21]  Hermano I Krebs,et al.  Rehabilitation robotics: pilot trial of a spatial extension for MIT-Manus , 2004, Journal of NeuroEngineering and Rehabilitation.

[22]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[23]  Hyung-Soon Park,et al.  Developing a whole-arm exoskeleton robot with hand opening and closing mechanism for upper limb stroke rehabilitation , 2009, 2009 IEEE International Conference on Rehabilitation Robotics.

[24]  M. Guadagnoli,et al.  Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning , 2004, Journal of motor behavior.

[25]  K. Scherer,et al.  Conscious emotional experience emerges as a function of multilevel, appraisal-driven response synchronization , 2008, Consciousness and Cognition.

[26]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[27]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Björn W. Schuller,et al.  Emotion representation, analysis and synthesis in continuous space: A survey , 2011, Face and Gesture 2011.

[29]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[30]  P. Ekman An argument for basic emotions , 1992 .

[31]  S. Filipović,et al.  Psychophysiology: Human Behavior and Physiological Response (4th ed.) , 2001 .

[32]  Nikolaos G. Tsagarakis,et al.  "Soft" Exoskeletons for Upper and Lower Body Rehabilitation - Design, Control and Testing , 2007, Int. J. Humanoid Robotics.

[33]  Jiping He,et al.  RUPERT: An exoskeleton robot for assisting rehabilitation of arm functions , 2008, 2008 Virtual Rehabilitation.

[34]  Fatih Ozkul,et al.  Evaluation of elbow joint proprioception with RehabRoby: a pilot study. , 2012 .

[35]  M.E. Miiller Why Some Emotional States Are Easier to be Recognized Than Others: A thorough data analysis and a very accurate rough set classifier , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[36]  C. Burgar,et al.  MIME robotic device for upper-limb neurorehabilitation in subacute stroke subjects: A follow-up study. , 2006, Journal of rehabilitation research and development.

[37]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[38]  Daniel Schneider,et al.  Computer-Based Learning Environments with Emotional Agents , 2009 .

[39]  R. Riener,et al.  Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. , 2011, Journal of rehabilitation research and development.

[40]  J. Veltman,et al.  Physiological workload reactions to increasing levels of task difficulty. , 1998, Ergonomics.

[41]  Jonghwa Kim,et al.  Bimodal Emotion Recognition using Speech and Physiological Changes , 2007 .

[42]  D. Carroll,et al.  Heart rate and oxygen consumption during active psychological challenge: the effects of level of difficulty. , 1986, Psychophysiology.

[43]  D. Reinkensmeyer,et al.  Technologies and combination therapies for enhancing movement training for people with a disability , 2012, Journal of NeuroEngineering and Rehabilitation.

[44]  Thierry Pun,et al.  Valence-arousal evaluation using physiological signals in an emotion recall paradigm , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[45]  Alicia Casals,et al.  Auto-adaptive robot-aided therapy using machine learning techniques , 2014, Comput. Methods Programs Biomed..

[46]  S. Leonhardt,et al.  A survey on robotic devices for upper limb rehabilitation , 2014, Journal of NeuroEngineering and Rehabilitation.

[47]  Robert Riener,et al.  ARMin III --arm therapy exoskeleton with an ergonomic shoulder actuation , 2009 .

[48]  Engin Masazade,et al.  Clustering of emotional states under different task difficulty levels for the robot-assisted rehabilitation system-RehabRoby , 2014, 2014 11th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[49]  Wanhui Wen,et al.  Emotion Recognition Based on Multi-Variant Correlation of Physiological Signals , 2014, IEEE Transactions on Affective Computing.

[50]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[51]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[52]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[53]  A. Mehrabian Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament , 1996 .

[54]  M. Munih,et al.  Psychophysiological Measurements in a Biocooperative Feedback Loop for Upper Extremity Rehabilitation , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[55]  Jordi Vallverdú,et al.  Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence , 2009 .

[56]  J. Andreassi Psychophysiology: Human Behavior and Physiological Response , 1980 .

[57]  J.C. Perry,et al.  Upper-Limb Powered Exoskeleton Design , 2007, IEEE/ASME Transactions on Mechatronics.

[58]  Duygun Erol Barkana,et al.  Upper-Extremity Rehabilitation Robot RehabRoby: Methodology, Design, Usability and Validation , 2013 .

[59]  Duygun Erol Barkana,et al.  A Robot‐Assisted Rehabilitation System – RehabRoby , 2013 .

[60]  M. Ohsuga,et al.  Assessment of phasic work stress using autonomic indices. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[61]  Javier de Lope,et al.  Static and Dynamic Multi-Robot Coverage with Grammatical Evolution Guided by Reinforcement and Semantic Rules , 2012 .

[62]  Antonio Frisoli,et al.  Development of a new exoskeleton for upper limb rehabilitation , 2009, 2009 IEEE International Conference on Rehabilitation Robotics.

[63]  Su-Lim Tan,et al.  A biometric signature based system for improved emotion recognition using physiological responses from multiple subjects , 2010, 2010 8th IEEE International Conference on Industrial Informatics.

[64]  David B. Kaber,et al.  Emotional State Classification in Patient–Robot Interaction Using Wavelet Analysis and Statistics-Based Feature Selection , 2013, IEEE Transactions on Human-Machine Systems.

[65]  Egon L. van den Broek,et al.  Affective Man-Machine Interface: Unveiling Human Emotions through Biosignals , 2009, BIOSTEC.

[66]  M. Dawson,et al.  The electrodermal system , 2007 .

[67]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[68]  W. Suess,et al.  The effects of psychological stress on respiration: a preliminary study of anxiety and hyperventilation. , 1980, Psychophysiology.

[69]  J. Gross,et al.  Mechanisms of Virtual Reality Exposure Therapy: The Role of the Behavioral Activation and Behavioral Inhibition Systems , 2005, Applied psychophysiology and biofeedback.

[70]  Bijan Shirinzadeh,et al.  Kinematics Analysis of 6-DOF Parallel Micro-Manipulators with Offset U-Joints: A Case Study , 2012, Int. J. Intell. Mechatronics Robotics.