Integration of stereo vision and optical flow by using an energy-minimization approach

A cooperative method is proposed in which image intensity (brightness) and optical-flow information are integrated into a single stereo technique by modeling the input data as coupled Markov random fields (MRF’s). The Bayesian probabilistic estimation method and the MRF–Gibbs equivalence theory are used to integrate the optical flow and the gray-level intensity information to obtain an energy function that will explicitly represent the depth discontinuity and occlusion constraints on the solution. This energy function involves the similarity in intensity (or edge orientation) and the optical flow between corresponding sites of the left and right images as well as the smoothness constraint on the disparity solution. If a simple MRF is used to model the data, the energy function will yield a poor disparity by smoothing across object boundaries, particularly when occluding objects are present. We use optical-flow information to indicate object boundaries (depth discontinuities) and occluded regions, in order to improve the disparity solution in occluded regions. A stochastic relaxation algorithm (simulated annealing) is used to find a favorable disparity solution by minimization of the energy equation.

[1]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[2]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[3]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[4]  Martin D. Levine,et al.  Computer determination of depth maps , 1973, Comput. Graph. Image Process..

[5]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  T. Poggio,et al.  Visual Integration and Detection of Discontinuities: The Key Role of Intensity Edges , 1987 .

[7]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[8]  Tomaso Poggio,et al.  A Theory of Human Stereo Vision , 1977 .

[9]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[10]  Ramakant Nevatia,et al.  Segment-based stereo matching , 1985, Comput. Vis. Graph. Image Process..

[11]  F. R. Hansen,et al.  Image segmentation using simple markov field models , 1982, Computer Graphics and Image Processing.

[12]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[13]  Ellen C. Hildreth,et al.  Computations Underlying the Measurement of Visual Motion , 1984, Artif. Intell..

[14]  Stephen T. Barnard,et al.  A Stochastic Approach to Stereo Vision , 1986, AAAI.

[15]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[16]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[17]  Yi Liu,et al.  Stereo vision correspondence using a multichannel graph matching technique , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[18]  T. Poggio,et al.  III-Posed problems early vision: from computational theory to analogue networks , 1985, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[19]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[21]  W. Eric L. Grimson,et al.  Computational Experiments with a Feature Based Stereo Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Hans P. Moravec.  Rover Visual Obstacle Avoidance , 1981, IJCAI.

[23]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[24]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[25]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[26]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[27]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[28]  Thomas O. Binford,et al.  Depth from Edge and Intensity Based Stereo , 1981, IJCAI.

[29]  C Koch,et al.  Analog "neuronal" networks in early vision. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Robert L. Henderson,et al.  Automatic Stereo Reconstruction Of Man-Made Targets , 1979, Other Conferences.