Reconstruction of structured scenes from two uncalibrated images

This paper investigates a practical heuristics method for reconstruction of structured scenes from two uncalibrated images. The method is based on an initial estimation of principal homographies from 2D initial point matches that may contain some outliers, and the homographies are refined recursively by incorporating the supporting matches of both points and lines on principal space surfaces. Then epipolar geometry is recovered directly from the refined homographies and cameras are calibrated from three orthogonal vanishing points and the recovered infinite homography. There are several points of novelty. First, a simple homography-guided method for matching line segments between two views is proposed. Second, under the assumption of zero-skew, the cameras are auto-calibrated with all the four intrinsic parameters varying between the two views. The advantages of the method is that it can build more realistic models with minimal human interactions, and it also allows us to reconstruct more visible surfaces on the detected planes than traditional methods which can only reconstruct the overlapping parts, since the homography provides a one-to-one mapping of points and lines between different views. Extensive experiments with real images illustrate the validity and advantages of the proposed method.

[1]  Patrick Gros,et al.  Matching and Clustering: Two Steps Toward Automatic Object Modeling in Computer Vision , 1995, Int. J. Robotics Res..

[2]  Olivier D. Faugeras,et al.  Determining the fundamental matrix with planes: instability and new algorithms , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Roberto Cipolla,et al.  PhotoBuilder-3D models of architectural scenes from uncalibrated images , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[4]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[5]  Andrew Zisserman,et al.  New Techniques for Automated Architectural Reconstruction from Photographs , 2002, ECCV.

[6]  Andrew Zisserman,et al.  Model selection for automated reconstruction from multiple views , 2002, BMVC.

[7]  G. F. McLean,et al.  Vanishing Point Detection by Line Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Andrew Zisserman,et al.  Automatic reconstruction of piecewise planar models from multiple views , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Richard Szeliski,et al.  Geometrically Constrained Structure from Motion: Points on Planes , 1998, SMILE.

[10]  Peter F. Sturm,et al.  Critical motion sequences for monocular self-calibration and uncalibrated Euclidean reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Jana Kosecka,et al.  Extraction, matching and pose recovery based on dominant rectangular structures , 2003, HLK.

[12]  Adrien Bartoli,et al.  Constrained Structure and Motion From Multiple Uncalibrated Views of a Piecewise Planar Scene , 2003, International Journal of Computer Vision.

[13]  Paul Debevec,et al.  Modeling and Rendering Architecture from Photographs , 1996, SIGGRAPH 1996.

[14]  É. Vincent,et al.  Detecting planar homographies in an image pair , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[15]  Stephen J. Maybank,et al.  On plane-based camera calibration: A general algorithm, singularities, applications , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[17]  Andrew Zisserman,et al.  Resolving ambiguities in auto–calibration , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[18]  B. Caprile,et al.  Using vanishing points for camera calibration , 1990, International Journal of Computer Vision.

[19]  Andrew Zisserman,et al.  Model Selection for Automated Architectural Reconstruction from Multiple Views , 2002 .

[20]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[21]  Jana Kosecka,et al.  Extraction, matching and pose recovery based on dominant rectangular structures , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[22]  Luc Van Gool,et al.  Planar homologies as a basis for grouping and recognition , 1998, Image Vis. Comput..

[23]  Andrew Zisserman,et al.  Combining scene and auto-calibration constraints , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  F. A. Heuvel.  VANISHING POINT DETECTION FOR ARCHITECTURAL PHOTOGRAMMETRY , 1998 .

[25]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[26]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[27]  Cordelia Schmid,et al.  Automatic line matching across views , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  J. Hothmer.  Book reviewInternational archives of photogrammetry and remote sensing: ISPRS, Editor S. Murai: volume 27 part A, Tokyo-Japan-Japan 1989 , 1989 .

[29]  Peter F. Sturm,et al.  3D Modelling Using Geometric Constraints: A Parallelepiped Based Approach , 2002, ECCV.

[30]  Roberto Cipolla,et al.  Automatic 3D Modelling of Architecture , 2000, BMVC.

[31]  Harry Shum,et al.  A linear algorithm for camera self-calibration, motion and structure recovery for multi-planar scenes from two perspective images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).