Robust Hand Pose Regression Using Convolutional Neural Networks

Hand pose estimation is useful for several human-computer interaction applications, like sign language recognition, the identification of more complex behaviors such as hand gestures and interaction in virtual reality applications. In this work, we propose a system which is able to predict the 2D hand joints using a monocular color camera. To do that, we propose to use a 3D hand tracking sensor for collecting ground truth information that is projected to the camera image plane. We present a novel pipeline that leverages deep learning techniques for hand pose estimation. The proposed Convolutional Neural Networks (CNN) is able to infer the joints of the hand from an image without the need of any additional sensor.