Guided optimisation through classification and regression for hand pose estimation

Proposes hand pose estimation using a combination of model optimisation and discriminative methods which allows tracking to be performed at over 40 frames per second using a single CPU thread.Introduces a residual error regression for hand pose estimation, learning from mistakes in model optimisation.A method of training, which captures system response and user variance, allowing supervised feedback for joint refinement.Extensive quantitative and qualitative evaluation including additional datasets and comparison against multiple state of the art approaches. This paper presents an approach to hand pose estimation that combines discriminative and model-based methods to leverage the advantages of both. Randomised Decision Forests are trained using real data to provide fast coarse segmentation of the hand. The segmentation then forms the basis of constraints applied in model fitting, using an efficient projected GaussSeidel solver, which enforces temporal continuity and kinematic limitations. However, when fitting a generic model to multiple users with varying hand shape, there is likely to be residual errors between the model and their hand. Also, local minima can lead to failures in tracking that are difficult to recover from. Therefore, we introduce an error regression stage that learns to correct these instances of optimisation failure. The approach provides improved accuracy over the current state of the art methods, through the inclusion of temporal cohesion and by learning to correct from failure cases. Using discriminative learning, our approach performs guided optimisation, greatly reducing model fitting complexity and radically improves efficiency. This allows tracking to be performed at over 40 frames per second using a single CPU thread.