Shield fields: modeling and capturing 3D occluders

We describe a unified representation of occluders in light transport and photography using shield fields: the 4D attenuation function which acts on any light field incident on an occluder. Our key theoretical result is that shield fields can be used to decouple the effects of occluders and incident illumination. We first describe the properties of shield fields in the frequency-domain and briefly analyze the "forward" problem of efficiently computing cast shadows. Afterwards, we apply the shield field signal-processing framework to make several new observations regarding the "inverse" problem of reconstructing 3D occluders from cast shadows -- extending previous work on shape-from-silhouette and visual hull methods. From this analysis we develop the first single-camera, single-shot approach to capture visual hulls without requiring moving or programmable illumination. We analyze several competing camera designs, ultimately leading to the development of a new large-format, mask-based light field camera that exploits optimal tiled-broadband codes for light-efficient shield field capture. We conclude by presenting a detailed experimental analysis of shield field capture and 3D occluder reconstruction.

[1]  J. Goodman.  Introduction to Fourier optics , 1969 .

[2]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[3]  Harry Shum,et al.  Plenoptic sampling , 2000, SIGGRAPH.

[4]  David Salesin,et al.  Surface light fields for 3D photography , 2000, SIGGRAPH.

[5]  R. Accorsi,et al.  Optimal coded aperture patterns for improved SNR in nuclear medicine imaging , 2001 .

[6]  Wojciech Matusik,et al.  Opacity light fields: interactive rendering of surface light fields with view-dependent opacity , 2003, I3D '03.

[7]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[9]  Frédo Durand,et al.  A frequency analysis of light transport , 2005, SIGGRAPH '05.

[10]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[11]  Eero P. Simoncelli,et al.  Range estimation by optical differentiation. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.

[13]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[14]  Herbert E. Ives,et al.  A Camera for Making Parallax Panoramagrams , 1928 .

[15]  P. Perona,et al.  Shadow carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[17]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[18]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[19]  F. Durand,et al.  Antialiasing for Automultiscopic 3 D Displays , 2006 .

[20]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[21]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[22]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[23]  Marc Levoy,et al.  Veiling glare in high dynamic range imaging , 2007, SIGGRAPH 2007.

[24]  Ren Ng.  Fourier Slice Photography , 2005 .

[25]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[26]  Kun Zhou,et al.  Precomputed shadow fields for dynamic scenes , 2005, SIGGRAPH 2005.

[27]  Takeo Kanade,et al.  Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  Ravi Ramamoorthi,et al.  A Fourier Theory for Cast Shadows , 2004, ECCV.

[29]  François X. Sillion,et al.  Fast calculation of soft shadow textures using convolution , 1998, SIGGRAPH.

[30]  T. M. Cannon,et al.  Coded aperture imaging with uniformly redundant arrays. , 1978, Applied optics.

[31]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[32]  Ravi Ramamoorthi,et al.  A first-order analysis of lighting, shading, and shadows , 2007, TOGS.

[33]  David Salesin,et al.  Spatio-angular resolution tradeoffs in integral photography , 2006, EGSR '06.

[34]  G. Lippmann.  Epreuves reversibles donnant la sensation du relief , 1908 .