Exploiting Discontinuities in Optical Flow

Most optical flow estimation techniques have substantial difficulties dealing with flow discontinuities. Methods which simultaneously detect flow boundaries and use the detected boundaries to aid in flow estimation can produce significantly improved results. Current approaches to implementing these methods still have important limitations, however. We demonstrate three such problems: errors due to the mixture of image properties across boundaries, an intrinsic ambiguity in boundary location when only short sequences are considered, and difficulties insuring that the motion of a boundary aids in flow estimation for the surface to which it is attached without corrupting the flow estimates for the occluded surface on the other side. The first problem can be fixed by basing flow estimation only on image changes at edges. The second requires an analysis of longer time intervals. The third can be aided by using a boundary detection mechanism which classifies the sides of boundaries as occluding and occluded at the same time as the boundaries are detected.

[1]  T. Poggio,et al.  Visual Integration and Detection of Discontinuities: The Key Role of Intensity Edges , 1987 .

[2]  W. Thompson,et al.  Hierarchical Estimation of Spatial Properties from Motion , 1984 .

[3]  George A. Kaplan,et al.  Kinetic disruption of optical texture: The perception of depth at an edge , 1969 .

[4]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Christof Koch,et al.  Computing optical flow in resistive networks and in the primate visual system , 1989, [1989] Proceedings. Workshop on Visual Motion.

[6]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[7]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[8]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[9]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[10]  Tomaso Poggio,et al.  Cooperative computation of stereo disparity , 1988 .

[11]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[12]  Brian G. Schunck,et al.  Image Flow Segmentation and Estimation by Constraint Line Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Michael J. Black,et al.  A model for the detection of motion over time , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[14]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[15]  William B. Thompson,et al.  Qualitative constraints for structure-from-motion , 1992, CVGIP Image Underst..

[16]  Hans-Hellmut Nagel,et al.  Optical flow estimation and the interaction between measurement errors at adjacent pixel positions , 1995, International Journal of Computer Vision.

[17]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[18]  Eric Dubois,et al.  Bayesian Estimation of Motion Vector Fields , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Guy Lindsay Scott,et al.  Local and Global Interpretation of Moving Images , 1988 .

[20]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Valdis Berzins,et al.  Dynamic Occlusion Analysis in Optical Flow Fields , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Ellen C. Hildreth,et al.  Measurement of Visual Motion , 1984 .

[23]  Shmuel Peleg,et al.  Computing two motions from three frames , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[24]  William B. Thompson,et al.  Structure-from-motion based on information at surface boundaries , 1992, Biological Cybernetics.

[25]  William B. Thompson,et al.  Lower-Level Estimation and Interpretation of Visual Motion , 1981, Computer.

[26]  Joseph K. Kearney,et al.  Optical Flow Estimation: An Error Analysis of Gradient-Based Methods with Local Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[28]  William B. Thompson,et al.  Analysis of Accretion and Deletion at Boundaries in Dynamic Scenes , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Mubarak Shah,et al.  Optimal Corner Detector , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[31]  K. Prazdny,et al.  Detection of binocular disparities , 2004, Biological Cybernetics.

[32]  William B. Thompson,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2009 .

[33]  Jin Luo,et al.  Computing motion using analog and binary resistive networks , 1988, Computer.

[34]  Allen M. Waxman,et al.  Convected activation profiles and the measurement of visual motion , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  G. B. Smith,et al.  Preface to S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” , 1987 .