The Joint Image Handbook

Given multiple perspective photographs, point correspondences form the "joint image", effectively a replica of three dimensional space distributed across its two-dimensional projections. This set can be characterized by multilinear equations over image coordinates, such as epipolar and trifocal constraints. We revisit in this paper the geometric and algebraic properties of the joint image, and address fundamental questions such as how many and which multilinearities are necessary and/or sufficient to determine camera geometry and/or image correspondences. The new theoretical results in this paper answer these questions in a very general setting and, in turn, are intended to serve as a "handbook" reference about multilinearities for practitioners.

[1]  T. Willmore.  Algebraic Geometry , 1973, Nature.

[2]  Donal O'Shea,et al.  Ideals, varieties, and algorithms - an introduction to computational algebraic geometry and commutative algebra (2. ed.) , 1997, Undergraduate texts in mathematics.

[3]  PAUL D. SAMPSON,et al.  Fitting conic sections to "very scattered" data: An iterative refinement of the bookstein algorithm , 1982, Comput. Graph. Image Process..

[4]  Patrizia M. Gianni,et al.  Gröbner Bases and Primary Decomposition of Polynomial Ideals , 1988, J. Symb. Comput..

[5]  Rekha R. Thomas,et al.  The Euclidean Distance Degree of an Algebraic Variety , 2013, Foundations of Computational Mathematics.

[6]  Jean Ponce,et al.  Trinocular Geometry Revisited , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  A. Heyden,et al.  Algebraic properties of multilinear constraints , 1997 .

[9]  Minas E. Spetsakis,et al.  Structure from motion using line correspondences , 1990, International Journal of Computer Vision.

[10]  Adam Boocher,et al.  Free Resolutions and Sparse Determinantal Ideals , 2011, 1111.0279.

[11]  Rekha R. Thomas,et al.  A Hilbert Scheme in Computer Vision , 2011, Canadian Journal of Mathematics.

[12]  Bill Triggs,et al.  Matching constraints and the joint image , 1995, Proceedings of IEEE International Conference on Computer Vision.

[13]  A. Heyden.  Tensorial properties of multiple view constraints , 2000 .

[14]  Narendra Ahuja,et al.  Motion and Structure from Line Correspondences; Closed-Form Solution, Uniqueness, and Optimization , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[16]  Adrien Bartoli,et al.  Structure-from-motion using lines: Representation, triangulation, and bundle adjustment , 2005, Comput. Vis. Image Underst..

[17]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[18]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[19]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Amnon Shashua,et al.  Algebraic Functions For Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[22]  G. Laman.  On graphs and rigidity of plane skeletal structures , 1970 .

[23]  Michael Kalkbrener,et al.  On the Stability of Gröbner Bases Under Specializations , 1997, J. Symb. Comput..

[24]  B. Sturmfels,et al.  Combinatorial Commutative Algebra , 2004 .

[25]  Qing Liu,et al.  Algebraic Geometry and Arithmetic Curves , 2002 .

[26]  Takayuki Hibi,et al.  Binomial edge ideals and conditional independence statements , 2009, Adv. Appl. Math..

[27]  Frederik Schaffalitzky,et al.  How hard is 3-view triangulation really? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  E SpetsakisMinas,et al.  Structure from motion using line correspondences , 1990 .

[29]  Richard I. Hartley,et al.  Reconstruction from Projections Using Grassmann Tensors , 2004, International Journal of Computer Vision.

[30]  Herwig Hauser,et al.  The Hironaka theorem on resolution of singularities (Or: A proof we always wanted to understand) , 2003 .