Combined displacement estimation and segmentation in image sequences

This paper addresses the problem of displacement field estimation and segmentation in image sequences. Emerging from the Bayesian paradigm, we derive an objective function yielding the MAP estimate with respect to some model assumptions. It can be interpreted as a measure for the estimates' explanation of the image data regularized by our prior assumptions on the estimates. The observation model we impose, considers experimental studies of the displaced frame difference and decovered regions. It involves some unknown parameters. The a priori is modelled by a coupled Gibbs/Markov random field. Optimization is performed via deterministic relaxation in a multiscale pyramid maintaining the structure of the algorithm in all pyramid levels. Iteratively, the unknown parameters of the observation model are estimated. The relaxation procedure tests only a small number of likely displacement-label candidates at each site. The relationship of regularization weights in the pyramid is thoroughly investigated. Simulation results with complex natural scenes demonstrate the good performance of the algorithm.