A Novel Model to Generate Heterogeneous and Realistic Time-Series Data for Post-Stroke Rehabilitation Assessment

The application of machine learning-based tele-rehabilitation faces the challenge of limited availability of data. To overcome this challenge, data augmentation techniques are commonly employed to generate synthetic data that reflect the configurations of real data. One such promising data augmentation technique is the Generative Adversarial Network (GAN). However, GANs have been found to suffer from mode collapse, a common issue where the generated data fails to capture all the relevant information from the original dataset. In this paper, we aim to address the problem of mode collapse in GAN-based data augmentation techniques for post-stroke assessment. We applied the GAN to generate synthetic data for two post-stroke rehabilitation datasets and observed that the original GAN suffered from mode collapse, as expected. To address this issue, we propose a Time Series Siamese GAN (TS-SGAN) that incorporates a Siamese network and an additional discriminator. Our analysis, using the longest common sub-sequence (LCSS), demonstrates that TS-SGAN generates data uniformly for all elements of two testing datasets, in contrast to the original GAN. To further evaluate the effectiveness of TS-SGAN, we encode the generated dataset into images using Gramian Angular Field and classify them using ResNet-18. Our results show that TS-SGAN achieves a significant accuracy increase of classification accuracy (35.2%-42.07%) for both selected datasets. This represents a substantial improvement over the original GAN.

[1]  M. Carrozza,et al.  Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: a systematic review , 2022, Journal of neuroengineering and rehabilitation.

[2]  K. Mcdonald-Maier,et al.  Predicting the Internal Knee Abduction Impulse During Walking Using Deep Learning , 2022, Frontiers in Bioengineering and Biotechnology.

[3]  Frank J. Wouda,et al.  Activities of Daily Living-Based Rehabilitation System for Arm and Hand Motor Function Retraining After Stroke , 2022, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Issam Boukhennoufa,et al.  Wearable sensors and machine learning in post-stroke rehabilitation assessment: A systematic review , 2022, Biomed. Signal Process. Control..

[5]  Danda B. Rawat,et al.  On the Performance of Generative Adversarial Network by Limiting Mode Collapse for Malware Detection Systems , 2021, Sensors.

[6]  K. Mcdonald-Maier,et al.  A comprehensive evaluation of state-of-the-art time-series deep learning models for activity-recognition in post-stroke rehabilitation assessment , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[7]  Ke Gong,et al.  Feature Refinement and Filter Network for Person Re-Identification , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  K. Mcdonald-Maier,et al.  Improving the activity recognition using GMAF and transfer learning in post-stroke rehabilitation assessment , 2021, 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI).

[9]  Truyen Tran,et al.  Catastrophic forgetting and mode collapse in GANs , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[10]  Anthony G. Constantinides,et al.  Body Sensor Networking, Design and Algorithms , 2020 .

[11]  Carlos Fernandez-Granda,et al.  Towards data-driven stroke rehabilitation via wearable sensors and deep learning , 2020, MLHC.

[12]  Xiaomin Song,et al.  Time Series Data Augmentation for Deep Learning: A Survey , 2020, IJCAI.

[13]  Alexandre Bernardino,et al.  Opportunities of a Machine Learning-based Decision Support System for Stroke Rehabilitation Assessment , 2020, ArXiv.

[14]  Cheolkon Jung,et al.  Fully Convolutional Siamese Fusion Networks for Object Tracking , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[15]  Jonas Adler,et al.  Banach Wasserstein GAN , 2018, NeurIPS.

[16]  Ali Borji,et al.  Pros and Cons of GAN Evaluation Measures , 2018, Comput. Vis. Image Underst..

[17]  Vincent Dumoulin,et al.  Generative Adversarial Networks: An Overview , 2017, 1710.07035.

[18]  Saeid Sanei,et al.  Triaxial rehabilitative data analysis incorporating matching pursuit , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[19]  Charles A. Sutton,et al.  VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning , 2017, NIPS.

[20]  Bernhard Schölkopf,et al.  AdaGAN: Boosting Generative Models , 2017, NIPS.

[21]  M. Maier,et al.  Upper Limb Outcome Measures Used in Stroke Rehabilitation Studies: A Systematic Literature Review , 2016, PloS one.

[22]  Gustavo Carneiro,et al.  Learning Local Image Descriptors with Deep Siamese and Triplet Convolutional Networks by Minimizing Global Loss Functions , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Aaron C. Courville,et al.  Generative Adversarial Networks , 2014, 1406.2661.

[26]  N. Lannin,et al.  Telerehabilitation services for stroke. , 2013, The Cochrane database of systematic reviews.

[27]  Peter Oster,et al.  Rehabilitation after stroke. , 2011, Deutsches Arzteblatt international.

[28]  Michelle J. Johnson,et al.  Advances in upper limb stroke rehabilitation: a technology push , 2011, Medical & Biological Engineering & Computing.

[29]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[30]  Tim Johansson,et al.  Telerehabilitation in stroke care – a systematic review , 2011, Journal of telemedicine and telecare.

[31]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[32]  P. Lehoux,et al.  A systematic review of clinical outcomes, clinical process, healthcare utilization and costs associated with telerehabilitation , 2009, Disability and rehabilitation.

[33]  Dimitrios Gunopulos,et al.  Indexing multi-dimensional time-series with support for multiple distance measures , 2003, KDD '03.

[34]  Dimitrios Gunopulos,et al.  Discovering similar multidimensional trajectories , 2002, Proceedings 18th International Conference on Data Engineering.

[35]  R. Myerson Nash Equilibrium and the History of Economic Theory , 1999 .

[36]  Dimitrios Gunopulos,et al.  Finding Similar Time Series , 1997, PKDD.

[37]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[38]  D. Brillinger Time series - data analysis and theory , 1981, Classics in applied mathematics.

[39]  D. Carroll,et al.  A QUANTITATIVE TEST OF UPPER EXTREMITY FUNCTION. , 1965, Journal of chronic diseases.

[40]  K. Mcdonald-Maier,et al.  Encoding Sensors' Data into Images to Improve the Activity Recognition in Post Stroke Rehabilitation Assessment , 2022, ICPRAI.

[41]  Bo Zhang,et al.  Recent Advances of Generative Adversarial Networks in Computer Vision , 2019, IEEE Access.

[42]  Mihaela van der Schaar,et al.  Time-series Generative Adversarial Networks , 2019, NeurIPS.

[43]  Jeff Dean,et al.  Time Series , 2009, Encyclopedia of Database Systems.