The Geometry and Matching of Lines and Curves Over Multiple Views

This paper describes the geometry of imaged curves in two and three views. Multi-view relationships are developed for lines, conics and non-algebraic curves. The new relationships focus on determining the plane of the curve in a projective reconstruction, and in particular using the homography induced by this plane for transfer from one image to another. It is shown that given the fundamental matrix between two views, and images of the curve in each view, then the plane of a conic may be determined up to a two fold ambiguity, but local curvature of a curve uniquely determines the plane. It is then shown that given the trifocal tensor between three views, this plane defines a homography map which may be used to transfer a conic or the curvature from two views to a third. Simple expressions are developed for the plane and homography in each case.A set of algorithms are then described for automatically matching individual line segments and curves between images. The algorithms use both photometric information and the multiple view geometric relationships. For image pairs the homography facilitates the computation of a neighbourhood cross-correlation based matching score for putative line/curve correspondences. For image triplets cross-correlation matching scores are used in conjunction with line/curve transfer based on the trifocal geometry to disambiguate matches. Algorithms are developed for both short and wide baselines. The algorithms are robust to deficiencies in the segment extraction and partial occlusion.Experimental results are given for image pairs and triplets, for varying motions between views, and for different scene types. The methods are applicable to line/curve matching in stereo and trinocular rigs, and as a starting point for line/curve matching through monocular image sequences.

[1]  Gérard G. Medioni,et al.  Segmented shape descriptions from 3-view stereo , 1995, Proceedings of IEEE International Conference on Computer Vision.

[2]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[3]  Jan J. Gerbrands,et al.  Method for matching general stereo planar curves , 1995, Image Vis. Comput..

[4]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[5]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Radu Horaud,et al.  Stereo Correspondence Through Feature Grouping and Maximal Cliques , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ramakant Nevatia,et al.  Segment-based stereo matching , 1985, Comput. Vis. Graph. Image Process..

[9]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[10]  Andrew Blake,et al.  Robust estimation of surface curvature from deformation of apparent contours , 1991, Image Vis. Comput..

[11]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[12]  F. Bookstein.  Fitting conic sections to scattered data , 1979 .

[13]  Olivier D. Faugeras,et al.  Curve-based stereo: figural continuity and curvature , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Tony P. Pridmore,et al.  Geometrical Modeling from Multiple Stereo Views , 1989, Int. J. Robotics Res..

[15]  Régis Vaillant.  Using Occluding Contours for 3D Object Modeling , 1990, ECCV.

[16]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 2004, International Journal of Computer Vision.

[17]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[18]  Zhengyou Zhang,et al.  Token tracking in a cluttered scene , 1994, Image Vis. Comput..

[19]  D. Forsyth,et al.  Fast recognition using algebraic invariants , 1992 .

[20]  Nicholas Ayache,et al.  Efficient registration of stereo images by matching graph descriptions of edge segments , 1987, International Journal of Computer Vision.

[21]  B. Benhabib,et al.  3D-pose estimation from a quadratic-curved feature in two perspective views , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[22]  Luc Van Gool,et al.  Semi-local projective invariants for the recognition of smooth plane curves , 1996, International Journal of Computer Vision.

[23]  Minas E. Spetsakis,et al.  Structure from motion using line correspondences , 1990, International Journal of Computer Vision.

[24]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Songde Ma,et al.  Conics-based stereo, motion estimation, and pose determination , 1993, International Journal of Computer Vision.

[26]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Richard I. Hartley,et al.  A linear method for reconstruction from lines and points , 1995, Proceedings of IEEE International Conference on Computer Vision.

[28]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[30]  Long Quan,et al.  Conic Reconstruction and Correspondence From Two Views , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Cordelia Schmid,et al.  Automatic line matching across views , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Paolo Remagnino,et al.  Correlation techniques in adaptive template matching with uncalibrated cameras , 1995, Other Conferences.

[33]  Daphna Weinshall,et al.  Flexible Syntactic Matching of Curves and Its Application to Automatic Hierarchical Classification of Silhouettes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[35]  Roberto Cipolla,et al.  Qualitative surface shape from deformation of image curves , 2004, International Journal of Computer Vision.

[36]  Andrew Zisserman,et al.  Robust Parameterization and Computation of the Trifocal Tensor , 1996, BMVC.

[37]  A. Gruen.  ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[38]  Joseph L. Mundy,et al.  Representing Objects Using Topology , 1996, Object Representation in Computer Vision.

[39]  Cordelia Schmid,et al.  The Geometry and Matching of Curves in Multiple Views , 1998, ECCV.

[40]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[41]  Andrew Zisserman,et al.  Quadric reconstruction from dual-space geometry , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[42]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[43]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  James L. Crowley,et al.  Measurement and Integration of 3-D Structures By Tracking Edge Lines , 1990, ECCV.

[45]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[46]  Patrick Gros,et al.  Matching and Clustering: Two Steps Toward Automatic Object Modeling in Computer Vision , 1995, Int. J. Robotics Res..

[47]  Michael Brady,et al.  Stereo Matching Of Curves By Least Deformation , 1989, Proceedings. IEEE/RSJ International Workshop on Intelligent Robots and Systems '. (IROS '89) 'The Autonomous Mobile Robots and Its Applications.

[48]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[49]  Rama Chellappa,et al.  Hierarchical stereo and motion correspondence using feature groupings , 1995, International Journal of Computer Vision.

[50]  Ramakant Nevatia,et al.  Use of Monocular Groupings and Occlusion Analysis in a Hierarchical Stereo System , 1995, Comput. Vis. Image Underst..

[51]  Pascal Fua,et al.  Automatic Extraction of Generic House Roofs from High Resolution Aerial Imagery , 1996, ECCV.

[52]  Luc Van Gool,et al.  Semi-differential invariants for nonplanar curves , 1992 .

[53]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[54]  Cordelia Schmid,et al.  AUTOMATIC LINE MATCHING AND 3D RECONSTRUCTION OF BUILDINGS FROM MULTIPLE VIEWS , 1999 .

[55]  Olivier D. Faugeras,et al.  A theory of the motion fields of curves , 1993, International Journal of Computer Vision.

[56]  L. Gool,et al.  Semi-differential invariants , 1992 .

[57]  Roberto Cipolla,et al.  Motion from the frontier of curved surfaces , 1995, Proceedings of IEEE International Conference on Computer Vision.

[58]  James L. Crowley,et al.  Measurement and integration of 3-D structures by tracking edge lines , 1990, International Journal of Computer Vision.

[59]  John Porrill,et al.  Curve matching and stereo calibration , 1991, Image Vis. Comput..

[60]  Andrew Zisserman,et al.  A Case Against Epipolar Geometry , 1993, Applications of Invariance in Computer Vision.

[61]  Thierry Viéville,et al.  Canonical Representations for the Geometries of Multiple Projective Views , 1996, Comput. Vis. Image Underst..

[62]  Olivier D. Faugeras,et al.  A theory of self-calibration of a moving camera , 1992, International Journal of Computer Vision.

[63]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[64]  Valbonne CedexFRANCEfaugeras.  A theory of the motion elds of curves , 1993 .

[65]  Anders Heyden,et al.  Using conic correspondences in two images to estimate the epipolar geometry , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).