Robust estimation of motion vector fields with discontinuity and occlusion using local outliers rejection

This paper proposes the use of rejecting outliers locally to solve effectively two fundamental problems in estimating motion vector fields: motion vector discontinuity and occlusion. This approach does not introduce extra unknowns and parameters as other approaches, such as using the `line process' and the `occlusion process.' Since the objective function of this approach can be calculated locally for each pixel, a highly parallel implementation can be achieved. For the problem of motion vector discontinuity, we argue that since the outliers occur at the motion boundaries, we should reject them locally. We reject outliers at each pixel by thresholding with a threshold reference based on the surrounding neighbors. Thus, our approach is not sensitive to the amplitude of motion, and the normal smoothness assumptions can fully used on smooth areas. Also, since we apply outlier rejection to the motion model instead of to the measurements, we can reject outliers even when there are more outliers than non-outliers for a given pixel, such as at a moving corner for an 8-neighborhood system. The proposed approach has been extended to the problem of occlusion using three frames. Simulated annealing using a Gibbs sampler is used to solve the minimization problem. Experiments for synthetic and real image sequences have been conducted to illustrate the effectiveness of the proposed approach.