An IMU-compensated skeletal tracking system using Kinect for the upper limb

The Kinect device is being increasingly used in conjunction with rehabilitative actions. However, the use of Kinect as a skeletal tracking system requires several further modifications and technological breakthroughs. This study used inertial measurement units (IMUs) to complement skeletal tracking with the Kinect. The IMUs were used to compensate for errors in calculating shoulder and elbow joint angles detected by the Kinect device while the patients performed rehabilitation movements. Thirty normal participants were recruited, and their shoulder and elbow joint angles were recorded and calculated using the Kinect and IMUs while they moved during movement games. If movement with a larger measuring error was detected, the measurement was directed to the IMU to calculate the angle and calibrate the angles measured by the Kinect device. The mean percent errors of the Kinect measurements with respect to the IMU measurement at the shoulder joint during shoulder flexion and rotation at 90° of shoulder flexion were 15.08 ± 4.13 and 26.00 ± 7.41%, respectively. The mean percent errors of the Kinect measurements with respect to the IMU measurements at the elbow joint during shoulder flexion, shoulder rotation at 90° of shoulder abduction, and shoulder rotation at 90° of shoulder flexion were 12.92 ± 2.43, 17.75 ± 4.91, and 23.3 ± 7.01%, respectively. The mean percent errors for the participants’ shoulders in Game 2 and Game 3 were 15.47 ± 4.88 and 28.13 ± 8.51%, respectively, and the mean percent errors of the participants’ elbows in Game 3 were 55.62 ± 13.74%. The proposed method to calibrate the angles detected using the Kinect have a greater mean accuracy rate (84.58%) and a higher processing rate (10 ms/frame) than traditional methods that use only Kinect or IMUs. The proposed system increases the accuracy of movement detected by the Kinect device, and this increases the processing rate of the IMUs, thereby improving clinical practicality.

[1]  Alon Lerner,et al.  Enhanced interactive gaming by blending full-body tracking and gesture animation , 2010, SA '10.

[2]  John H Hollman,et al.  Reliability and accuracy of a goniometer mobile device application for video measurement of the functional movement screen deep squat test. , 2015, International journal of sports physical therapy.

[3]  B. Watanapa,et al.  Human gesture recognition using Kinect camera , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[4]  Yu-Che Cheng,et al.  Using MEMS-based inertial sensor with ankle foot orthosis for telerehabilitation and its clinical evaluation in brain injuries and total knee replacement patients , 2016 .

[5]  F. Horak,et al.  iTUG, a Sensitive and Reliable Measure of Mobility , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[6]  Norman R. Saunders,et al.  Use of a Virtual Reality Physical Ride-On Sailing Simulator as a Rehabilitation Tool for Recreational Sports and Community Reintegration: A Pilot Study , 2013, American journal of physical medicine & rehabilitation.

[7]  Hazim Kemal Ekenel,et al.  Game design for physical therapy and rehabilitation using Kinect , 2015, 2015 Medical Technologies National Conference (TIPTEKNO).

[8]  Ankit Chaudhary,et al.  Tracking of Fingertips and Centers of Palm Using KINECT , 2011, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation.

[9]  Xiaoyu Wu,et al.  Static Human Gesture grading based on Kinect , 2012, 2012 5th International Congress on Image and Signal Processing.

[10]  S.H. Yeo,et al.  Inertia sensor-based guidance system for upperlimb posture correction. , 2013, Medical engineering & physics.

[11]  Stacy J. Morris Bamberg,et al.  A feasibility study of an upper limb rehabilitation system using kinect and computer games , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  I-Ming Chen,et al.  Portable Posture Guiding System With Visual, Verbal Feedback for Upper Extremity , 2013 .

[13]  Yu. V. Filatov,et al.  Dynamic calibration method of inertial measurement units , 2015 .

[14]  Chen Feng,et al.  Upper limb motion tracking with the integration of IMU and Kinect , 2015, Neurocomputing.

[15]  Abdellatif Bey-Temsamani,et al.  2-Level error (drift) compensation for low-cost MEMS-based inertial measurement unit (IMU) , 2016 .

[16]  Kazushi Ikeda,et al.  Kinect-based posturography for in-home rehabilitation of balance disorders , 2015, APSIPA Transactions on Signal and Information Processing.

[17]  Tal Jarus,et al.  Virtual reality as balance rehabilitation for children with brain injury: A case study , 2013 .

[18]  Fei Liu,et al.  Motor impairment evaluation for upper limb in stroke patients on the basis of a microsensor , 2012, International journal of rehabilitation research. Internationale Zeitschrift fur Rehabilitationsforschung. Revue internationale de recherches de readaptation.

[19]  Tao Liu,et al.  Development of a wearable sensor system for quantitative gait analysis , 2009 .

[20]  Xiaoping Yun,et al.  Design, Implementation, and Experimental Results of a Quaternion-Based Kalman Filter for Human Body Motion Tracking , 2006, IEEE Trans. Robotics.