Simultaneous motion estimation and segmentation

We present a Bayesian framework that combines motion (optical flow) estimation and segmentation based on a representation of the motion field as the sum of a parametric field and a residual field. The parameters describing the parametric component are found by a least squares procedure given the best estimates of the motion and segmentation fields. The motion field is updated by estimating the minimum-norm residual field given the best estimate of the parametric field, under the constraint that motion field be smooth within each segment. The segmentation field is updated to yield the minimum-norm residual field given the best estimate of the motion field, using Gibbsian priors. The solution to successive optimization problems are obtained using the highest confidence first (HCF) or iterated conditional mode, (ICM) optimization methods. Experimental results on real video are shown.

[1]  Paul S. Heckbert,et al.  Fundamentals of Texture Mapping and Image Warping , 1989 .

[2]  J. A. Brewer,et al.  Visual interaction with overhauser curves and surfaces , 1977, SIGGRAPH '77.

[3]  Christoph Stiller,et al.  Object-oriented video coding employing dense motion fields , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Norbert Diehl,et al.  Object-oriented motion estimation and segmentation in image sequences , 1991, Signal Process. Image Commun..

[6]  J. A. Parker,et al.  Comparison of Interpolating Methods for Image Resampling , 1983, IEEE Transactions on Medical Imaging.

[7]  Eric Dubois,et al.  Bayesian Estimation of Motion Vector Fields , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  A. Murat Tekalp,et al.  An algorithm for simultaneous motion estimation and scene segmentation , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  George Wolberg,et al.  Digital image warping , 1990 .

[11]  Stephen E. Reichenbach,et al.  Two-Parameter Cubic Convolution For Image Reconstruction , 1989, Other Conferences.

[12]  Siu-Leong Iu.  Robust Estimation of Motion Vector Fields with Discontinuity and Occlusion Using Local Outliers Rejection , 1995, J. Vis. Commun. Image Represent..

[13]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[14]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[15]  Edmund Taylor Whittaker.  XVIII.—On the Functions which are represented by the Expansions of the Interpolation-Theory , 1915 .

[16]  Robert A. Schowengerdt,et al.  Image reconstruction by parametric cubic convolution , 1982, Comput. Graph. Image Process..

[17]  L. Rabiner,et al.  A digital signal processing approach to interpolation , 1973 .

[18]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[19]  E. Catmull,et al.  A CLASS OF LOCAL INTERPOLATING SPLINES , 1974 .

[20]  P. Anandan,et al.  Accurate computation of optical flow by using layered motion representations , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[21]  Michael Unser,et al.  Fast B-spline Transforms for Continuous Image Representation and Interpolation , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  M. Hötter,et al.  Image segmentation based on object oriented mapping parameter estimation , 1988 .

[23]  Patrick Bouthemy,et al.  Segmentation and estimation of image motion by a robust method , 1994, Proceedings of 1st International Conference on Image Processing.

[24]  M. J. Cunningham,et al.  A Function Space Model for Digital Image Sampling and Its Application in Image Reconstruction , 1990, Comput. Vis. Graph. Image Process..

[25]  E. Maeland.  On the comparison of interpolation methods. , 1988, IEEE transactions on medical imaging.

[26]  오승준.  [서평]「Digital Video Processing」 , 1996 .

[27]  Siu-Leong Iu,et al.  Robust estimation of motion vector fields with discontinuity and occlusion using local outliers rejection , 1993, Other Conferences.

[28]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[30]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[31]  Arun N. Netravali,et al.  Reconstruction filters in computer-graphics , 1988, SIGGRAPH.

[32]  Donald E. Troxel,et al.  Transformation Between Continuous and Discrete Representations of Images: A Perceptual Approach , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.