Recognizing pair-activities by causality analysis

In this article, beyond solo-activity analysis for single object, we study the more complicated pair-activity recognition problem by exploring the relationship between two active objects based on their trajectory clues obtained from video sensor. Our contributions are three-fold. First, we design two sets of features for representing the pair-activities encoded as length-variable trajectory pairs. One set characterizes the strength of causality between two trajectories, for example, the causality ratio and feedback ratio based on the Granger Causality Test (GCT), and another set describes the style of causality between two trajectories, for example, the sampled frequency responses of the digital filter with these two trajectories as the input and output discrete signals respectively. These features along with conventional velocity and position features of a trajectory-pair are essentially of multi-modalities, and may be greatly different in scales and importance. To make full use of them, we then develop a novel feature fusing procedure to learn the coefficients for weighting these features by maximizing the discriminating power measured by weighted correlation. Finally, we collected a pair-activity database of five popular categories, each of which consists of about 170 instances. The extensive experiments on this database validate the effectiveness of the designed features for pair-activity representation, and also demonstrate that the proposed feature fusing procedure significantly boosts the pair-activity classification accuracy.