Motion-based video segmentation using continuation method and robust cost functions

We propose a new approach to spatial segmentation of video sequences that is based on motion attributes. The approach, similarly to some previous efforts, uses Markov random field models and maximum a posteriori probability estimation.Our approach is novel in three ways. First, we propose a general formation for the joint motion estimation. Secondly, instead of the usual quadratic models we propose a robust estimation criterion that eliminates the impact of outliers on the estimates. Thirdly, since solving the segmentation problem directly in the space of discrete labels is difficult, we opt for a continuation method over a Gaussian pyramid. Thus, the estimation process starts as a motion estimation and then slowly converges towards a motion-based segmentation by 'hardening' the smoothness constraint. The final result is a quasi-segmentation, i.e., the estimated vector field is continuous but almost peicewise constant, and must undergo subsequent quantization. We show experimental results on two natural image sequences; the resolution quasi-segmentations clearly extract moving objects. The method may serve as an initial stage for joint motion estimation and segmentation, or may produce final segmentations if suitable post- processing is applied.