Multiple motion analysis: in spatial or in spectral domain?

In this paper, we compare the effects of multiple motions in spatial and spectral representations of an image sequence. We describe multiple motions in both domains and establish a comparison regarding their inherent properties when discretized. Though the spectral model provides us with an explicit description of both occlusion and transparency, it turns out that its resolution is very limited. We show that the spatial domain represented by the spatio-temporal derivatives has superior resolution properties and is thus more appropriate for the treatment of occlusion. We present an algorithm which based on an initial estimate of the number of motions uses the shift-and-subtract technique to localize occlusion boundaries and to track their movement in occlusion sequences. The same technique is used to distinguish occlusion from transparency and to decompose transparency scenes into multi-layers.

[1]  Weichuan Yu,et al.  Three dimensional orientation signatures with conic kernel filtering for multiple motion analysis , 2003, Image Vis. Comput..

[2]  Weichuan Yu,et al.  Detection and characterization of multiple motion points , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shmuel Peleg,et al.  A Three-Frame Algorithm for Estimating Two-Component Image Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[6]  David J. Fleet.  Measurement of image velocity , 1992 .

[7]  V. B. Uvarov,et al.  Special Functions of Mathematical Physics: A Unified Introduction with Applications , 1988 .

[8]  Bernd Jähne,et al.  Spatio-Temporal Image Processing , 1993, Lecture Notes in Computer Science.

[9]  Hans Knutsson,et al.  Filtering and reconstruction in image processing , 1984 .

[10]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Weichuan Yu,et al.  3D-orientation signatures with conic kernel filtering for multiple motion analysis , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Steven S. Beauchemin,et al.  On the Fourier Properties of Discontinuous Motion , 2000, Journal of Mathematical Imaging and Vision.

[14]  Rolf P. Würtz,et al.  Object Recognition Robust Under Translations, Deformations, and Changes in Background , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[16]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Joachim Weickert,et al.  Räumlich-zeitliche Berechnung des optischen Flusses mit nichtlinearen flußabhängigen Glattheitstermen , 1999, DAGM-Symposium.

[18]  David J. Fleet,et al.  Probabilistic Detection and Tracking of Motion Boundaries , 2000, International Journal of Computer Vision.

[19]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Brian G. Schunck,et al.  Image Flow Segmentation and Estimation by Constraint Line Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[22]  A. Yuille,et al.  A model for the estimate of local image velocity by cells in the visual cortex , 1990, Proceedings of the Royal Society of London. B. Biological Sciences.

[23]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Luc Van Gool,et al.  Determination of Optical Flow and its Discontinuities using Non-Linear Diffusion , 1994, ECCV.

[25]  Keith Langley,et al.  Computational analysis of non-Fourier motion , 1994, Vision Research.

[26]  Tai Sing Lee,et al.  Image Representation Using 2D Gabor Wavelets , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[28]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[29]  Steven A. Shafer,et al.  Moment and Hypergeometric Filters for High Precision Computation of Focus, Stereo and Optical Flow , 1994, International Journal of Computer Vision.

[30]  Josef Kittler,et al.  Estimation of complex multimodal motion: an approach based on robust statistics and Hough transform , 1994, Image Vis. Comput..

[31]  J. Bigun,et al.  Optimal Orientation Detection of Linear Symmetry , 1987, ICCV 1987.

[32]  Michael S. Langer,et al.  Dimensional Analysis of Image Motion , 2001, ICCV.

[33]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[34]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[35]  Suzanne Beauchemin,et al.  The Frequency Structure of 1D Occluding Image Signals , 2000 .

[36]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[37]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[38]  Kenji Mase,et al.  Unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Josef Kittler,et al.  A Gradient-Based Method for General Motion Estimation and Segmentation , 1993, J. Vis. Commun. Image Represent..

[40]  David J. Fleet,et al.  Design and Use of Linear Models for Image Motion Analysis , 2000, International Journal of Computer Vision.

[41]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Minoru Asada,et al.  MDL-Based Segmentation and Motion Modeling in a Long Image Sequence of Scene with Multiple Independently Moving Objects , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  J. L. Barronxy.  On the Fourier Properties of Discontinuous VisualMotionS , 2000 .

[44]  Patrick Bouthemy,et al.  A Maximum Likelihood Framework for Determining Moving Edges , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Steven S. Beauchemin,et al.  The Frequency Structure of One-Dimensional Occluding Image Signals , 2000, IEEE Trans. Pattern Anal. Mach. Intell..