When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision?

The complete set of measurements that could ever be used by a passive 3D vision algorithm is the plenoptic function or light-field. We give a concise characterization of when the light-field of a Lambertian scene uniquely determines its shape and, conversely, when the shape is inherently ambiguous. In particular, we show that stereo computed from the light-field is ambiguous if and only if the scene is radiating light of a constant intensity (and color, etc.) over an extended region.

[1]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[2]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[3]  Richard S. Weiss,et al.  Reconstruction of Surfaces from Profiles , 1987, ICCV 1987.

[4]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[5]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[6]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Olivier D. Faugeras,et al.  Computing differential properties of 3-D shapes from stereoscopic images without 3-D models , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Shree K. Nayar,et al.  Stereo in the presence of specular reflection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[12]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[13]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[14]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[15]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  G. Cheung.  Visual Hull Construction, Alignment and Refinement Across Time , 2001 .

[17]  Takeo Kanade,et al.  A Characterization of Inherent Stereo Ambiguities , 2001, ICCV.

[18]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[19]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[20]  Peter N. Belhumeur,et al.  A Bayesian approach to binocular steropsis , 1996, International Journal of Computer Vision.

[21]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.