Semi-automatic video object segmentation based on hierarchy optical flow

Abstract In the new MPEG-4 video coding standard, the semi-automatic video segmentation plays a key role insupporting object-oriented coding and enabling content-based functionalities. A novel hierarchy optical flow basedsemi-automatic video segmentation method is presented in this paper. The proposed segmentation method containsspatial and temporal segmentation. For the spatial segmentation, a point-based graphic user interface (PBGUI) ispresented, with which the user can input easily, and then active contour model and tracking bug algorithm are applied toprecisely define the video object of interest to be segmented. With the result of spatial segmentation, the temporalsegmentation involves non-rigid object contour tracking and rigid object whole-tracking by hierarchy optical flowalgorithm based on the Lucas-Kanade algorithm. And the tracking point selection algorithm is proposed to greatlyimprove the tracking performance in the rigid object whole-tracking. The experimental results show that the proposedalgorithm can precisely segment video objects from video streams.Keywords: MPEG-4, optical flow, video segmentation, object-oriented coding