Estimation theoretic approach for robust predictive motion field segmentation

Segmenting a block motion field into two regions was proposed to reduce prediction errors in block-based video coding algorithm. Because the segmentation cannot be computed at the decoder without the frame to code, it was also proposed to compute a predicted segmentation from the previous coded frames. The performance of such an approach highly depends on the difference between the predicted and the true segmentation. Although under translational motion the predicted and true segmentation are identical, coding noise introduces distortions in the past images which result in a predicted segmentation with errors along the boundaries between the different regions. Small deviations from the hypothesis of translational motion also have a similar effect. In this paper, we propose to take into account the uncertainty in the predicted segmentation and thereby improve the motion compensated prediction. This is accomplished by optimizing the predicted values of the pixels in a segmented region with respect to the segmentation uncertainty. By also taking into account the uncertainty on the region boundaries when the segmentation is computed, we further improve the motion compensated prediction. The proposed algorithm significantly reduces the MSE and bit rate when compared to a standard block matching algorithm.