Video Annotation by Motion Interpretation Using Optical Flow Streams

Abstract A new approach to automatic annotation of video sequences by dominant camera motion interpretation is presented. Unlike other approaches, we separate the optical flow into two categories— singular and non-singular —which, as we show, is a more natural way of classification for the purpose of dominant camera motion interpretation. We show that identification of patterns created by such natural categories, which can be observed from the measured optical flow, can help focus the interpretation of dominant camera motion in video segments. For robust detection of such natural patterns, we propose the computation of optical flow streams (OFS) from the video data and analyze the OFS for the extraction of dominant motion content in the video segments. The advantage of the proposed approach is its robustness in the extraction of dominant motion content in a video segment. We demonstrate this on a variety of real video sequences by generating the automatic motion annotation of the video frames and comparing it with manual motion annotation.