Robust dynamic hand gesture recognition system with sparse steric haar-like feature for human robot interaction

Hand gesture is an effective and natural way for human-robot interaction (HRI). This paper presents a robust dynamic hand gesture recognition system with a RGB-D sensor. In order to automatically recognize hand gesture from color and depth sequences, where noise and occlusion are common problems, we extract steric Haar-like features to robustly represent the complicated spatial information of the hand. A novel feature selection approach, which takes the advantage of class separability measure, is employed to effectively ferret out the most discriminative features. We also use sparse coding method to encode these features so that it is less prone to over-fitting even when only limited amount of training data are available. Generally speaking, spare Steric Haar-like (SSH) features are efficient to compute by using the self-padding integral volume, in addition to the advantage of robustness to noise and occlusion. These crucial features significantly improve the performance of tracking and classification. Experiments with a public dynamic hand gesture dataset and a self-built hand gesture dataset show the superiority of the proposed system compared with the state-of-the-art approaches.