Human Action Recognition: Learning Sparse Basis Units from Trajectory Subspace

ABSTRACT Human action recognition from image sequences is a challenging issue in the computer vision society. Each action can be decomposed into a few intrinsic parts that concisely model the action based on the architecture of the visual cortex. To recognize human actions, we model a video as a sequence of visual words in which every change in the human pose corresponds to a “word.” This step is followed by sparse coding feature learning with the aim of transforming low-level descriptors of visual words into richer representations of intermediate complexity, called mid-level features, similar to what happens in the primary visual cortex (area V1). In the next stage, due to extracted temporal information, a concise and informative topic from phrase trajectories is introduced that takes into account the advantages of an effective locality-constrained linear coding (LLC) algorithm. The goal of this coding method is to represent input vectors as approximate linear combinations of a small number of “basis vectors,” which are described as high-level features, utilizing the locality constraints to project each descriptor into its local-coordinate system. Recognizing an action leads to matching these active basis sequences within the larger video sequence. Although it is common to represent each video as a histogram of visual words, this type of representation discards the temporal information inherent to actions. To maintain the temporal order of actions in sequence, the hidden Markov model (HMM) is utilized. The main contribution of this study is to elicit mid- and high-level features in conjunction with incorporating the temporal ordering constraints between the basis units in a real-time manner. We evaluate our methodology on the KTH, Weismann, and UCF-sports human action datasets. Experimental results demonstrate that the feature learning in spatial and temporal domains can improve the final action recognition in terms of speed, accuracy, and interpretability compared to the state-of-art methods.