Constrained, globally optimal, multi-frame motion estimation

We address the problem of estimating the relative motion between the frames of a video sequence. In comparison with the commonly applied pairwise image registration methods, we consider global consistency conditions for the overall multi-frame motion estimation problem, which is more accurate. We review the recent work on this subject and propose an optimal framework, which can apply the consistency conditions as both hard constraints in the estimation problem, or as soft constraints in the form of stochastic (Bayesian) priors. The framework is applicable to virtually any motion model and enables us to develop a robust approach, which is resilient against the effects of outliers and noise. The effectiveness of the proposed approach is confirmed by a super-resolution application on synthetic and real data sets

[1]  Walter Bender,et al.  Salient video stills: content and context preserved , 1993, MULTIMEDIA '93.

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[4]  Yaser Sheikh,et al.  AN ACCUMULATIVE FRAMEWORK FOR THE ALIGNMENT OF AN IMAGE SEQUENCE , 2003 .

[5]  Harpreet S. Sawhney,et al.  Is Super-Resolution with Optical Flow Feasible? , 2002, ECCV.

[6]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[7]  BrownLisa Gottesfeld.  A survey of image registration techniques , 1992 .

[8]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[9]  Michael Elad,et al.  Multiframe demosaicing and super-resolution of color images , 2006, IEEE Transactions on Image Processing.

[10]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[11]  Venu Madhav Govindu.  Lie-algebraic averaging for globally consistent motion estimation , 2004, CVPR 2004.

[12]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[13]  Alan S. Willsky,et al.  Algebraic Structure and Finite Dimensional Nonlinear Estimation , 1978 .

[14]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[15]  Harpreet S. Sawhney,et al.  Robust Video Mosaicing through Topology Inference and Local to Global Alignment , 1998, ECCV.

[16]  Tsuhan Chen,et al.  Correlation Based Search Algorithms for Motion Estimation , 1999 .

[17]  Shmuel Peleg,et al.  Efficient super-resolution and applications to mosaics , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.