Kinect Joints Correction Using Optical Flow for Weightlifting Videos

To ease a coach in weightlifting training, automatic weightlifting pattern evaluation is required. In order to do that, the motion tracking process is always needed. Kinect sensor is one of the popular sensors for that. However, there is a problem with skeleton created by the Kinect sensor because of self-occlusion. Hence, in this paper, we develop a joint correction process for 3 types of joints including hands, feet, and knees since these joints are sometimes provided incorrectly. However, we only correct these joints in the "the first pull to the transition from the first to the second pull" (first-step) in snatch, and clean and jerk weightlifting and "the turnover under the barbell to the catch phase" (second-step) in clean and jerk weightlifting. This is because miscalculation occurs only in these steps. We utilized fast cross-correlation and the Lucas-Kanade algorithm to compute the optical flow of the consecutive frames. From that, we then correct the joints if they are misplaced from the predicted joints. Our system provides better joints and more preferable to human eyes than the original Kinect skeleton.