A Deep Learning Approach to Prevent Problematic Movements of Industrial Workers Based on Inertial Sensors

Nowadays, manufacturing industries still face difficulties applying traditional Work-related MusculoSkeletal Disorders (WMSDs) risk assessment methods due to the high effort required by a continuous data collection when using observational methods. An interesting solution is to adopt Inertial Measurement Units (IMUs) to automate the data collection, thus supporting occupational health professionals. In this paper, we propose a deep learning approach to predict human motion based on IMU data with the goal of preventing industrial worker problematic movements that can arise during repetitive actions. The proposed system includes an initial Madgwick filter to merge the raw inertial tri-axis sensor data into a single angle orientation time series. Then, a Machine Learning (ML) algorithm is trained with the obtained time series, allowing to build a forecasting model. The effectiveness of the developed system was validated by using an open-source dataset composed of different motions for the upper body collected in a laboratory environment, aiming to monitor the abduction/adduction angle of the arm. Firstly, distinct ML algorithms were compared for a single angle orientation time series prediction, including: three Long Short-Term Memory (LSTM) methods - a one layer, a stacked layer and a Sequence to Sequence (Seq2Seq) model; and three non deep learning methods - a Multiple Linear Regression, a Random Forest and a Support Vector Machine. The best results were provided by the Seq2Seq LSTM model, which was further evaluated for WMSD prevention by considering 11 human subject datasets and two evaluation procedures (single person and multiple person training and testing). Overall, interesting results were achieved, particularly for multiple person evaluation, where the proposed Seq2Seq LSTM has shown an excellent capability to anticipate problematic movements.

[1]  Bernd Markert,et al.  A Comparison of Three Neural Network Approaches for Estimating Joint Angles and Moments from Inertial Measurement Units , 2021, Sensors.

[2]  Jianlong Xu,et al.  FM-GRU: A Time Series Prediction Method for Water Quality Based on seq2seq Framework , 2021, Water.

[3]  Huu Du Nguyen,et al.  Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management , 2020, Int. J. Inf. Manag..

[4]  Simone A. Ludwig.  Investigation of Orientation Estimation of Multiple IMUs , 2020, Unmanned Syst..

[5]  Pratik Chaudhari,et al.  Evaluation of Machine Learning Models for Classifying Upper Extremity Exercises Using Inertial Measurement Unit-Based Kinematic Data , 2020, IEEE Journal of Biomedical and Health Informatics.

[6]  Witold Pedrycz,et al.  A survey on machine learning for data fusion , 2020, Inf. Fusion.

[7]  Oscar Mayora-Ibarra,et al.  Choosing the Best Sensor Fusion Method: A Machine-Learning Approach , 2020, Sensors.

[8]  Antonios Danelakis,et al.  Automated Real-time Anomaly Detection in Human Trajectories using Sequence to Sequence Networks , 2019, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[9]  A. Cereatti,et al.  Accuracy of the Orientation Estimate Obtained Using Four Sensor Fusion Filters Applied to Recordings of Magneto-Inertial Sensors Moving at Three Rotation Rates , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[10]  Petros Maragos,et al.  LSTM-based Network for Human Gait Stability Prediction in an Intelligent Robotic Rollator , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[11]  Akbar Siami Namin,et al.  A Comparison of ARIMA and LSTM in Forecasting Time Series , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[12]  Francesco Caputo,et al.  IMU-Based Motion Capture Wearable System for Ergonomic Assessment in Industrial Environment , 2018, Advances in Human Factors in Wearable Technologies and Game Design.

[13]  Florentino Serranheira,et al.  Neck and upper limb musculoskeletal symptoms in assembly line workers of an automotive industry in Portugal , 2017 .

[14]  O. Sarbishei,et al.  On the accuracy improvement of low-power orientation filters using IMU and MARG sensor arrays , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[15]  Martin Stepnicka,et al.  Forecasting seasonal time series with computational intelligence: On recent methods and the potential of their combinations , 2013, Expert Syst. Appl..

[16]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[17]  Miguel Rio,et al.  Symbiotic filtering for spam email detection , 2011, Expert Syst. Appl..

[18]  Paulo Cortez,et al.  Data Mining with Neural Networks and Support Vector Machines Using the R/rminer Tool , 2010, ICDM.

[19]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[20]  Hugo Gamboa,et al.  Self-Similarity Matrix of Morphological Features for Motion Data Analysis in Manufacturing Scenarios , 2021, BIOSIGNALS.

[21]  Mingxi Liu,et al.  A novel cryptocurrency price trend forecasting model based on LightGBM , 2020 .

[22]  Hugo Gamboa,et al.  Explaining the Ergonomic Assessment of Human Movement in Industrial Contexts , 2020, BIOSIGNALS.

[23]  Xiaosong Shu,et al.  The Prediction of Dam Displacement Time Series Using STL, Extra-Trees, and Stacked LSTM Neural Network , 2020, IEEE Access.