Augmentation of Segmented Motion Capture Data for Improving Generalization of Deep Neural Networks

This paper presents a method for augmenting the motion capture trajectories to improve generalization performance of recurrent long short-term memory (LSTM) neural networks. The presented algorithm is based on the interpolation of existing time series and can be applied only to segmented or easy-to-segment data due to the possibility of blending similar motion trajectories that are not significantly time-shifted. The paper shows the results of the classification efficiency with and without augmentation for two publicly available databases: Multimodal Kinect-IMU Dataset and National Chiao Tung University Multisensor Fitness Dataset. The former contains the data representing separate human computer interaction gestures, while the latter comprises the data of unsegmented series of body exercises. As a result of using the presented algorithm, the classification accuracy increased by approximately 11% points for the first dataset and 8% points for the second one.

[1]  Dana Kulic,et al.  Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks , 2017, ICMI.

[2]  Luping Xu,et al.  Image Denoising Using Hybrid Contourlet and Bandelet Transforms , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[3]  Yueting Zhuang,et al.  Sparse motion bases selection for human motion denoising , 2015, Signal Process..

[4]  Meng Wang,et al.  Sign language recognition based on adaptive HMMS with data augmentation , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[5]  Peter M. Quesada,et al.  Wavelet-based noise removal for biomechanical signals: a comparative study , 2000, IEEE Transactions on Biomedical Engineering.

[6]  R. Venkatesh Babu,et al.  Human gait recognition using depth camera: a covariance based approach , 2012, ICVGIP '12.

[7]  Dong Seog Han,et al.  Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network , 2018, Sensors.

[8]  Juan Pablo Wachs,et al.  A Human-Centered Approach to One-Shot Gesture Learning , 2017, Front. Robot. AI.

[9]  Yu-Chee Tseng,et al.  A Comprehensive Multisensor Dataset Employing RGBD Camera, Inertial Sensor and Web Camera , 2019, 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS).

[10]  Nasser Kehtarnavaz,et al.  Data Augmentation in Deep Learning-Based Fusion of Depth and Inertial Sensing for Action Recognition , 2019, IEEE Sensors Letters.

[11]  Pavlo Molchanov,et al.  Hand gesture recognition with 3D convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Hong Liu,et al.  Spatial-Temporal Data Augmentation Based on LSTM Autoencoder Network for Skeleton-Based Human Action Recognition , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[13]  Juan Pablo Wachs,et al.  Biomechanical-Based Approach to Data Augmentation for One-Shot Gesture Recognition , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[14]  Changhe Tu,et al.  Classification of gait anomalies from kinect , 2018, The Visual Computer.

[15]  Thien Huynh-The,et al.  Encoding Pose Features to Images With Data Augmentation for 3-D Action Recognition , 2020, IEEE Transactions on Industrial Informatics.

[16]  Emma Fortune,et al.  Validity of using tri-axial accelerometers to measure human movement - Part I: Posture and movement detection. , 2014, Medical engineering & physics.

[17]  Matteo Gadaleta,et al.  IDNet: Smartphone-based Gait Recognition with Convolutional Neural Networks , 2016, Pattern Recognit..

[18]  Kebin Jia,et al.  A New and Effective Image Retrieval Method Based on Combined Features , 2007, Fourth International Conference on Image and Graphics (ICIG 2007).

[19]  Tzuu-Hseng S. Li,et al.  Motion Imitation and Augmentation System for a Six Degrees of Freedom Dual-Arm Robot , 2019, IEEE Access.

[20]  Héctor Pomares,et al.  Kinect=IMU? Learning MIMO Signal Mappings to Automatically Translate Activity Recognition Systems across Sensor Modalities , 2012, 2012 16th International Symposium on Wearable Computers.

[21]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..