Learning Bodily and Temporal Attention in Protective Movement Behavior Detection

For people with chronic pain, the assessment of protective behavior during physical functioning is essential to understand their subjective pain-related experiences (e.g., fear and anxiety toward pain and injury) and how they deal with such experiences (avoidance or reliance on specific body joints), with the ultimate goal of guiding intervention. Advances in deep learning (DL) can enable the development of such intervention. Using the EmoPain MoCap dataset, we investigate how attention-based DL architectures can be used to improve the detection of protective behavior by capturing the most informative temporal and body configurational cues characterizing specific movements and the strategies used to perform them. We propose an end-to-end deep learning architecture named BodyAttentionNet (BANet). BANet is designed to learn temporal and bodily parts that are more informative to the detection of protective behavior. The approach addresses the variety of ways people execute a movement (including healthy people) independently of the type of movement analyzed. Through extensive comparison experiments with other state-of-the-art machine learning techniques used with motion capture data, we show statistically significant improvements achieved by using these attention mechanisms. In addition, the BANet architecture requires a much lower number of parameters than the state of the art for comparable if not higher performances.

[1]  M. S. Hane Aung,et al.  Automatic recognition of fear-avoidance behavior in chronic pain physical rehabilitation , 2014, PervasiveHealth.

[2]  Maja Pantic,et al.  The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset , 2016, IEEE Transactions on Affective Computing.

[3]  Pascal Thibault,et al.  The influence of communication goals and physical demands on different dimensions of pain behavior , 2006, Pain.

[4]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[5]  Shaohan Hu,et al.  QualityDeepSense: Quality-Aware Deep Learning Framework for Internet of Things Applications with Sensor-Temporal Attention , 2018, EMDL@MobiSys.

[6]  Temitayo A. Olugbade,et al.  The relationship between guarding, pain, and emotion , 2019, Pain reports.

[7]  Ana Tajadura-Jiménez,et al.  Motivating people with chronic pain to do physical activity: opportunities for technology design , 2014, CHI.

[8]  Francis J. Keefe,et al.  Development of an observation method for assessing pain behavior in chronic low back pain patients. , 1982 .

[9]  Richard Preuss,et al.  Movement variability in adults with low back pain during sit‐to‐stand‐to‐sit , 2018, Clinical biomechanics.

[10]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[11]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mohsen Shafizadeh,et al.  Movement coordination during sit-to-stand in low back pain people , 2016 .

[13]  Jeffrey M. Hausdorff,et al.  Potentials of Enhanced Context Awareness in Wearable Assistants for Parkinson's Disease Patients with the Freezing of Gait Syndrome , 2009, 2009 International Symposium on Wearable Computers.

[14]  Maja Pantic,et al.  The automatic detection of chronic pain-related expression : requirements , challenges and a multimodal dataset , 2014 .

[15]  P J Watson,et al.  Surface electromyography in the identification of chronic low back pain patients: the development of the flexion relaxation ratio. , 1997, Clinical biomechanics.

[16]  Didier Stricker,et al.  Introducing a New Benchmarked Dataset for Activity Monitoring , 2012, 2012 16th International Symposium on Wearable Computers.

[17]  Nicolai Marquardt,et al.  Human Observer and Automatic Assessment of Movement Related Self-Efficacy in Chronic Pain: From Exercise to Functional Activity , 2020, IEEE Transactions on Affective Computing.

[18]  Nicholas D. Lane,et al.  Recurrent network based automatic detection of chronic pain protective behavior using MoCap and sEMG data , 2019, UbiComp.

[19]  Paul Lukowicz,et al.  Wearable Activity Tracking in Car Manufacturing , 2008, IEEE Pervasive Computing.

[20]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[21]  Daniel Roggen,et al.  Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations , 2016, SEMWEB.

[22]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[23]  Nicolai Marquardt,et al.  Bi-Modal Detection of Painful Reaching for Chronic Pain Rehabilitation Systems , 2014, ICMI.

[24]  P SumathiC.,et al.  Automatic Facial Expression Analysis A Survey , 2012 .

[25]  Nicolai Marquardt,et al.  Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[26]  Thomas Plötz,et al.  On attention models for human activity recognition , 2018, UbiComp.

[27]  B. Gelder,et al.  Why bodies? Twelve reasons for including bodily expressions in affective neuroscience , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[28]  Dana Kulic,et al.  Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks , 2017, ICMI.

[29]  Ming Zeng,et al.  Understanding and improving recurrent networks for human activity recognition by continuous attention , 2018, UbiComp.

[30]  J. Vlaeyen,et al.  Fear-avoidance and its consequences in chronic musculoskeletal pain: a state of the art , 2000, Pain.

[31]  Nadia Bianchi-Berthouze,et al.  Supporting Everyday Function in Chronic Pain Using Wearable Technology , 2017, CHI.

[32]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[33]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Nicholas D. Lane,et al.  Automatic Detection of Protective Behavior in Chronic Pain Physical Rehabilitation: A Recurrent Neural Network Approach , 2019, ArXiv.

[35]  G. Crombez,et al.  The experimental analysis of the interruptive, interfering, and identity-distorting effects of chronic pain. , 2016, Behaviour research and therapy.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Luca Benini,et al.  Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection , 2008, EWSN.

[38]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[39]  T. Sipko,et al.  The Effect of Chronic Pain Intensity on Sit-to-Stand Strategy in Patients With Herniated Lumbar Disks. , 2016, Journal of manipulative and physiological therapeutics.

[40]  B. Collett,et al.  Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment , 2006, European journal of pain.

[41]  Nicolai Marquardt,et al.  How Can Affect Be Detected and Represented in Technological Support for Physical Rehabilitation? , 2019, ACM Trans. Comput. Hum. Interact..

[42]  Markus Voelter,et al.  State of the Art , 1997, Pediatric Research.

[43]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[44]  Pietro Falco,et al.  A Human Action Descriptor Based on Motion Coordination , 2017, IEEE Robotics and Automation Letters.

[45]  Ana Tajadura-Jiménez,et al.  Go-with-the-Flow: Tracking, Analysis and Sonification of Movement and Breathing to Build Confidence in Activity Despite Chronic Pain , 2016, Hum. Comput. Interact..