Recurrence Matrices for Human Action Recognition

One important issue for action characterization consists of properly capturing temporally related information. In this work, recurrence matrices are explored as a way to represent action sequences. A recurrence matrix (RM) encodes all pair-wise comparisons of the frame-level descriptors. By its nature, a recurrence matrix can be regarded as a temporally holistic action representation, but it can hardly be used directly and some descriptor is therefore required to compactly summarize its contents. Two simple RM-level descriptors computed from a given recurrence matrix are proposed. A general procedure to combine a set of RM-level descriptors is presented. This procedure relies on a combination of early and late fusion strategies. Recognition performances indicate the proposed descriptors are competitive provided that enough training examples are available. One important finding is the significant impact on performance of both, which feature subsets are selected, and how they are combined, an issue which is generally overlooked.