Action Recognition Using Local Consistent Group Sparse Coding with Spatio-Temporal Structure

This paper presents a novel and efficient framework for human action recognition through integrating the local consistent group sparse representation with spatio-temporal structure of each video sequence. We firstly propose a sparse encoding scheme named local consistent group sparse coding (LCGSC) to generate the sparse representation of each video sequence. The novel encoding scheme takes global structural information of features belonging to one group into consideration as well as the local correlations between similar features. In order to incorporate the spatio-temporal structures, an average location (AL) model is proposed to describe the distribution of each visual word along the spatio-temporal coordinates on the basis of the obtained sparse codes. Eventually, each video sequence is jointly represented by the sparse representation and the spatio-temporal layouts which fully model its motion, appearance and spatio-temporal information. Our framework is computationally efficient and achieves comparable performance on the challenging datasets with state-of-the-art methods.