Continuous Estimation of Lower Limb Joint Angles From Multi-Stream Signals Based on Knowledge Tracing

Multi-stream signals are increasingly being used in robot-assisted rehabilitation training, where the timely and accurate prediction of a patient's motor intentions is frequently required to provide simultaneous and proportional control strategies. However, existing methods for motion intent prediction typically isolate signals from different modalities, resulting in a loss of signal traceability and interpretability. Therefore, we propose a multi-stream signal-fusion strategy based on knowledge tracing (MSKT). First, we collected surface electromyography, force myography, and vibroarthrography signals from the lower extremity and used different encoders to map these signals into a problem component, concept component, and identity token component. The process of human learning was then simulated using a knowledge-tracing strategy to reconstruct these components into new features. Finally, the reconstructed set of depth features was mapped to joint angles. We validated the performance of the proposed method using motion data from eight participants in different locomotion modes and compared the results to those of a temporal convolution network and long short-term memory networks. The results indicate that the MSKT achieves the lowest root-mean-squared error. This study demonstrates that the proposed knowledge-tracing-based strategy enables accurate continuous estimation for lower-limb exoskeleton rehabilitation robots.

[1]  Zhengkun Yi,et al.  Prediction of Contralateral Lower-Limb Joint Angles Using Vibroarthrography and Surface Electromyography Signals in Time-Series Network , 2023, IEEE Transactions on Automation Science and Engineering.

[2]  S. Lee,et al.  Estimating Ground Reaction Force and Center of Pressure Using Low-Cost Wearable Devices , 2021, IEEE Transactions on Biomedical Engineering.

[3]  M. Akay,et al.  Cancelable HD-SEMG Biometric Identification via Deep Feature Learning , 2021, IEEE Journal of Biomedical and Health Informatics.

[4]  Dacheng Tao,et al.  Gait Phase Classification for a Lower Limb Exoskeleton System Based on a Graph Convolutional Network Model , 2021, IEEE Transactions on Industrial Electronics.

[5]  Xiaoxia Zhou,et al.  A Novel Method for Detecting Misclassifications of the Locomotion Mode in Lower-Limb Exoskeleton Robot Control , 2022, IEEE Robotics and Automation Letters.

[6]  Oluwarotimi Williams Samuel,et al.  A Novel and Efficient Feature Extraction Method for Deep Learning Based Continuous Estimation , 2021, IEEE Robotics and Automation Letters.

[7]  Oluwarotimi Williams Samuel,et al.  A Bi-Directional LSTM Network for Estimating Continuous Upper Limb Movement From Surface Electromyography , 2021, IEEE Robotics and Automation Letters.

[8]  Chifu Yang,et al.  A Bipolar Myoelectric Sensor-Enabled Human-Machine Interface Based On Spinal Module Activations , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Chenyun Dai,et al.  Enhancing IoT Security via Cancelable HD-sEMG-Based Biometric Authentication Password, Encoded by Gesture , 2021, IEEE Internet of Things Journal.

[10]  Hui Xiong,et al.  Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting , 2020, AAAI.

[11]  Chenyun Dai,et al.  Cancelable HD-sEMG-Based Biometrics for Cross-Application Discrepant Personal Identification , 2020, IEEE Journal of Biomedical and Health Informatics.

[12]  Anoop Kant Godiyal,et al.  A New Force Myography-Based Approach for Continuous Estimation of Knee Joint Angle in Lower Limb Amputees and Able-Bodied Subjects , 2020, IEEE Journal of Biomedical and Health Informatics.

[13]  Hui Xiong,et al.  EKT: Exercise-Aware Knowledge Tracing for Student Performance Prediction , 2019, IEEE Transactions on Knowledge and Data Engineering.

[14]  Caitlin N. Teague,et al.  A Wearable, Multimodal Sensing System to Monitor Knee Joint Health , 2020, IEEE Sensors Journal.

[15]  Can Wang,et al.  Continuous Estimation of Knee Joint Angle Based on Surface Electromyography Using a Long Short-Term Memory Neural Network and Time-Advanced Feature , 2020, Sensors.

[16]  Li-Chen Fu,et al.  Deep Learning based Motion Prediction for Exoskeleton Robot Control in Upper Limb Rehabilitation , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[17]  Jun Morimoto,et al.  Exploiting Human and Robot Muscle Synergies for Human-in-the-loop Optimization of EMG-based Assistive Strategies , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[18]  Hong Cheng,et al.  Learning Physical Human–Robot Interaction With Coupled Cooperative Primitives for a Lower Exoskeleton , 2019, IEEE Transactions on Automation Science and Engineering.

[19]  Shiv Dutt Joshi,et al.  Force Myography Based Novel Strategy for Locomotion Classification , 2018, IEEE Transactions on Human-Machine Systems.

[20]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[21]  Jianwei Zhang,et al.  Hierarchical learning control with physical human-exoskeleton interaction , 2017, Inf. Sci..

[22]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[23]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[24]  Carlo Menon,et al.  Continuous Prediction of Finger Movements Using Force Myography , 2016 .

[25]  Guangjun Liu,et al.  An user-independent gesture recognition method based on sEMG decomposition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Leonidas J. Guibas,et al.  Deep Knowledge Tracing , 2015, NIPS.

[27]  Jun Morimoto,et al.  Bilinear Modeling of EMG Signals to Extract User-Independent Features for Multiuser Myoelectric Interface , 2013, IEEE Transactions on Biomedical Engineering.

[28]  Don A. Yungher,et al.  Surface muscle pressure as a measure of active and passive behavior of muscles during gait. , 2011, Medical engineering & physics.