Representing the finger-only topology for hand shape recognition

Automatic recognition of hand shapes in a moving image sequence requires the elements of hand tracking, feature extraction and classification. We have developed a robust tracking algorithm and a new hand shape representation technique that characterises the finger-only topology of the hand by adapting an existing technique from speech signal processing. The tracking algorithm determines the centre of the largest convex subset of the hand throughout an image sequence, using a combination of pattern matching and condensation algorithms. A hand shape feature represents the topological formation of the finger-only regions of the hand using a Linear Predictive Coding parameter set called cepstral coefficients. Feature extraction is performed on the polar dimensions of the hand region-of-interest, by tracking the finger-only region and extracting euclidean distances between the finger-only contour and the hand centre, which are then converted into cepstral coefficients. Experiments are conducted using mug-grabbing sequences to recognise 4 different hand shapes. Results demonstrate the robustness of hand tracking on cluttered backgrounds and the effectiveness of the hand shape representation technique on varying hand shapes.