Consistency of Silhouettes and Their Duals

Silhouettes provide rich information on three-dimensional shape, since the intersection of the associated visual cones generates the "visual hull", which encloses and approximates the original shape. However, not all silhouettes can actually be projections of the same object in space: this simple observation has implications in object recognition and multi-view segmentation, and has been (often implicitly) used as a basis for camera calibration. In this paper, we investigate the conditions for multiple silhouettes, or more generally arbitrary closed image sets, to be geometrically "consistent". We present this notion as a natural generalization of traditional multi-view geometry, which deals with consistency for points. After discussing some general results, we present a "dual" formulation for consistency, that gives conditions for a family of planar sets to be sections of the same object. Finally, we introduce a more general notion of silhouette "compatibility" under partial knowledge of the camera projections, and point out some possible directions for future research.

[1]  Aldo Laurentini,et al.  The visual hull for understanding shapes from contours: a survey , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[2]  Roberto Cipolla,et al.  Automatic 3D object segmentation in multiple views using volumetric graph-cuts , 2007, Image Vis. Comput..

[3]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[4]  R. Gardner.  Geometric Tomography: Parallel X-rays of planar convex bodies , 2006 .

[5]  W. Goldman.  PROJECTIVE GEOMETRY ON MANIFOLDS , 1988 .

[6]  Li Yi,et al.  Efficiently Determining Silhouette Consistency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Roberto Cipolla,et al.  Generalised Epipolar Constraints , 1996, ECCV.

[8]  Niloy J. Mitra,et al.  Shadow art , 2009, ACM Trans. Graph..

[9]  V. Howard,et al.  Unbiased Stereology: Three-Dimensional Measurement in Microscopy , 1998 .

[10]  W. Goldman.  Projective Geometry on Manifolds Lecture Notes (v.0.3) | Mathematics 748b Spring 1988 , 1988 .

[11]  Simon Baker,et al.  Visual hull construction, alignment and refinement for human kinematic modeling, motion tracking and rendering , 2003 .

[12]  Andrew W. Fitzgibbon,et al.  Parallax geometry of smooth surfaces in multiple views , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Van Hove,et al.  Silhouette-slice theorems , 1986 .

[14]  Edmond Boyer.  On Using Silhouettes for Camera Calibration , 2006, ACCV.

[15]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, International Journal of Computer Vision.

[16]  Roberto Cipolla,et al.  Affine Reconstruction of Curved Surfaces from Uncalibrated Views of Apparent Contours , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Jean Ponce,et al.  Carved Visual Hulls for Image-Based Modeling , 2006, ECCV.

[18]  Jorge Stolfi,et al.  Oriented Projective Geometry: A Framework for Geometric Computations , 2014 .

[19]  Roberto Cipolla,et al.  Reconstruction of sculpture from its profiles with unknown camera positions , 2004, IEEE Transactions on Image Processing.

[20]  G. M.,et al.  Projective Geometry , 1938, Nature.

[21]  Cordelia Schmid,et al.  On Pencils of Tangent Planes and the Recognition of Smooth 3D Shapes from Silhouettes , 2002, ECCV.

[22]  David W. Jacobs,et al.  Judging Whether Multiple Silhouettes Can Come from the Same Object , 2001, IWVF.

[23]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[24]  Joseph O'Rourke,et al.  Handbook of Discrete and Computational Geometry, Second Edition , 1997 .

[25]  Narendra Ahuja,et al.  Structure and Motion Estimation from Dynamic Silhouettes under Perspective Projection , 1999, International Journal of Computer Vision.

[26]  Martial Hebert,et al.  On Image Contours of Projective Shapes , 2014, ECCV.

[27]  J. H. Rieger.  Three-dimensional motion from fixed points of a deforming profile curve. , 1986, Optics letters.

[28]  Andrea Bottino,et al.  Introducing a New Problem: Shape-from-Silhouette when the Relative Positions of the Viewpoints is Unknown , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Evgueni A. A. Tevelev,et al.  Projective Duality and Homogeneous Spaces , 2005 .

[30]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[31]  Patrick Pérez,et al.  Multi-view Object Segmentation in Space and Time , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Daniel Cremers,et al.  Multiview Stereo and Silhouette Consistency via Convex Functionals over Convex Domains , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  K. Ball.  CONVEX BODIES: THE BRUNN–MINKOWSKI THEORY , 1994 .

[34]  R. Schneider.  Convex Bodies: The Brunn–Minkowski Theory: Minkowski addition , 1993 .

[35]  Paulo R. S. Mendonça,et al.  Epipolar geometry from profiles under circular motion , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[37]  David J. Kriegman,et al.  Structure and Motion from Images of Smooth Textureless Objects , 2004, ECCV.

[38]  Gert Vegter,et al.  In handbook of discrete and computational geometry , 1997 .

[39]  Jean Ponce,et al.  Projective Visual Hulls , 2007, International Journal of Computer Vision.

[40]  Jan J. Koenderink,et al.  Solid shape , 1990 .

[41]  Roberto Cipolla,et al.  Silhouette Coherence for Camera Calibration under Circular Motion , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Simon A. J. Winder,et al.  Object instance recognition using triplets of feature symbols , 2007 .