Human Action Recognition with Hierarchical Growing Neural Gas Learning

We propose a novel biologically inspired framework for the recognition of human full-body actions. First, we extract body pose and motion features from depth map sequences. We then cluster pose-motion cues with a two-stream hierarchical architecture based on growing neural gas (GNG). Multi-cue trajectories are finally combined to provide prototypical action dynamics in the joint feature space. We extend the unsupervised GNG with two labelling functions for classifying clustered trajectories. Noisy samples are automatically detected and removed from the training and the testing set. Experiments on a set of 10 human actions show that the use of multi-cue learning leads to substantially increased recognition accuracy over the single-cue approach and the learning of joint pose-motion vectors.