The Structure of Multiplicative Motions in Natural Imagery

A theoretical investigation of the frequency structure of multiplicative image motion signals is presented, e.g., as associated with translucency phenomena. Previous work has claimed that the multiplicative composition of visual signals generally results in the annihilation of oriented structure in the spectral domain. As a result, research has focused on multiplicative signals in highly specialized scenarios where highly structured spectral signatures are prevalent, or introduced a nonlinearity to transform the multiplicative image signal to an additive one. In contrast, in this paper, it is shown that oriented structure is present in multiplicative cases when natural domain constraints are taken into account. This analysis suggests that the various instances of naturally occurring multiple motion structures can be treated in a unified manner. As an example application of the developed theory, a multiple motion estimator previously proposed for translation, additive transparency, and occlusion is adapted to multiplicative image motions. This estimator is shown to yield superior performance over the alternative practice of introducing a nonlinear preprocessing step.

[1]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[2]  A. Mackay.  "Textures" , 1987 .

[3]  Keith Langley,et al.  Computational analysis of non-Fourier motion , 1994, Vision Research.

[4]  Weichuan Yu,et al.  Detection and characterization of multiple motion points , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  M. Shizawa,et al.  Principle of superposition: a common computational framework for analysis of multiple motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[6]  Edward H. Adelson,et al.  Ordinal characteristics of transparency. , 1990 .

[7]  Shmuel Peleg,et al.  A Three-Frame Algorithm for Estimating Two-Component Image Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[10]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[11]  Ivar Austvoll,et al.  A Study of the Yosemite Sequence Used as a Test Sequence for Estimation of Optical Flow , 2005, SCIA.

[12]  Weichuan Yu,et al.  Multiple motion analysis: in spatial or in spectral domain? , 2003, Comput. Vis. Image Underst..

[13]  Richard P. Wildes,et al.  Detecting Spatiotemporal Structure Boundaries: Beyond Motion Discontinuities , 2009, ACCV.

[14]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[15]  Azriel Rosenfeld,et al.  Robust regression methods for computer vision: A review , 1991, International Journal of Computer Vision.

[16]  Michael J. Black,et al.  Mixture models for optical flow computation , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[17]  D J Heeger,et al.  Model for the extraction of image flow. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[18]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[19]  Steven S. Beauchemin,et al.  The Frequency Structure of One-Dimensional Occluding Image Signals , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[21]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[22]  Weichuan Yu,et al.  Çöööòøøø Ëøöù Blockinøùöö Óó Øøø Ç Blockin , 2007 .

[23]  David J. Fleet.  Measurement of image velocity , 1992 .

[24]  Keith Langley,et al.  PII: S0042-6989(98)00093-5 , 1998 .

[25]  R. Wildes,et al.  Early spatiotemporal grouping with a distributed oriented energy representation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  T. Poggio,et al.  Visual hyperacuity: spatiotemporal interpolation in human vision , 1981, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[27]  Suzanne Beauchemin,et al.  The Frequency Structure of 1D Occluding Image Signals , 2000 .

[28]  Edward H. Adelson,et al.  Human-assisted motion annotation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ronald N. Bracewell,et al.  The Fourier Transform and Its Applications , 1966 .

[30]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[31]  F. Harris.  On the use of windows for harmonic analysis with the discrete Fourier transform , 1978, Proceedings of the IEEE.

[32]  D. Kersten.  Transparency and the cooperative computation of scene attributes , 1991 .

[33]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.