Match Propagation for Image-Based Modeling and Rendering

This paper presents a quasi-dense matching algorithm between images based on the match propagation principle. The algorithm starts from a set of sparse seed matches, then propagates to the neighboring pixels by the best-first strategy, and produces a quasi-dense disparity map. The quasi-dense matching aims at broad modeling and visualization applications which rely heavily on matching information. Our algorithm is robust to initial sparse match outliers due to the best-first strategy. It is efficient in time and space as it is only output sensitive. It handles half-occluded areas because of the simultaneous enforcement of newly introduced discrete 2D gradient disparity limit and the uniqueness constraint. The properties of the algorithm are discussed and empirically demonstrated. The quality of quasi-dense matching are validated through intensive real examples.

[1]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[2]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[3]  Olga Veksler,et al.  Disparity component matching for visual correspondence , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Long Quan,et al.  Robust dense matching using local and global geometric constraints , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[5]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[6]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[7]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[8]  B. Julesz,et al.  A disparity gradient limit for binocular fusion. , 1980, Science.

[9]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[10]  Zhengyou Zhang,et al.  A Progressive Scheme for Stereo Matching , 2000, SMILE.

[11]  Long Quan,et al.  Invariants of Six Points and Projective Reconstruction From Three Uncalibrated Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  G. M. Quenot.  The 'orthogonal algorithm' for optical flow detection using dynamic programming , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[13]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[14]  Aaron F. Bobick,et al.  Disparity-Space Images and Large Occlusion Stereo , 1994, ECCV.

[15]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[17]  Daniel Scharstein,et al.  Stereo vision for view synthesis , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Long Quan,et al.  Edge-constrained joint view triangulation for image interpolation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[20]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[21]  Steven M. Seitz,et al.  View morphing , 1996, SIGGRAPH.

[22]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[24]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  L. Falkenhagen.  Depth Estimation from Stereoscopic Image Pairs Assuming Piecewise Continuos Surfaces , 1995 .

[26]  Harry Shum,et al.  Motion estimation with quadtree splines , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Maxime Lhuillier,et al.  Efficient Dense Matching for Textured Scenes using Region Growing , 1998, BMVC.

[28]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[29]  Hans P. Morevec.  Towards automatic visual obstacle avoidance , 1977, IJCAI 1977.

[30]  Qian Chen,et al.  A volumetric stereo matching method: application to image-based modeling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[31]  Takeo Kanade,et al.  Constructing virtual worlds using dense stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[33]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[34]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[35]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[36]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[37]  Pascal Fua,et al.  Combining Stereo and Monocular Information to Compute Dense Depth Maps that Preserve Depth Discontinuities , 1991, IJCAI.

[38]  G. P. Otto,et al.  "Region-growing" algorithm for matching of terrain images , 1989, Image Vis. Comput..

[39]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[41]  Long Quan,et al.  Image interpolation by joint view triangulation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[42]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[43]  Ebroul Izquierdo,et al.  Image Analysis for 3D Modeling, Rendering, and Virtual View Generation , 1998, Comput. Vis. Image Underst..