Action Recognition by Motion Trajectory Decomposition

Recognition of human actions in a video acquired by a moving camera typically requires standard preprocessing steps such as motion compensation, moving object detection and object tracking. The errors from the motion compensation step propagate to the object detection stage, resulting in miss-detections, which further complicates the tracking stage, resulting in cluttered and incorrect tracks. Therefore, action recognition from a moving camera is considered very challenging. In this chapter, we discuss an approach which does not follow the standard steps, and accordingly avoids the aforementioned difculties. The approach is based on Lagrangian particle trajectories which are a set of dense trajectories obtained by advecting optical flow over time, thus capturing the ensemble motions of a scene. This is done in frames of unaligned video, and no object detection is required. Consequently, in order to handle the moving camera, a low-rank trajectory decomposition approach is employed, where the trajectories are decomposed into their camera-induced and object-induced components. Having obtained the relevant object motion trajectories, a compact set of chaotic invariant features are computed, which captures the characteristics of the trajectories. Finally, a SVM is employed to learn and recognize the human actions using the computed motion features.