Shadow cameras: Reciprocal views from illumination masks

Scene appearance from the point of view of a light source is called a reciprocal or dual view. Since there exists a large diversity in illumination, these virtual views may be non-perspective and multi-viewpoint in nature. In this paper, we demonstrate the use of occluding masks to recover these dual views, which we term shadow cameras. We first show how to render a single reciprocal scene view by swapping the camera and light source positions. We extend this technique for multiple views by building a virtual shadow camera array with static masks and a moving source. We also capture non-perspective views such as orthographic, cross-slit and a pushbroom variant, while introducing novel applications such as converting between camera projections and removing refractive and catadioptric distortions. Finally, since a shadow camera is artificial, we can manipulate any of its intrinsic parameters, such as camera skew, to create perspective distortions.

[1]  Pietro Perona,et al.  3D Reconstruction by Shadow Carving: Theory and Practical Evaluation , 2007, International Journal of Computer Vision.

[2]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[3]  Michael Werman,et al.  Vertical Parallax from Moving Shadows , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.

[5]  Steven M. Seitz,et al.  The Space of All Stereo Images , 2004, International Journal of Computer Vision.

[6]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[7]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[9]  Marc Levoy,et al.  Symmetric photography: exploiting data-sparseness in reflectance fields , 2006, EGSR '06.

[10]  Leonard McMillan,et al.  General Linear Cameras , 2004, ECCV.

[11]  Shree K. Nayar,et al.  Caustics of catadioptric cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Takeo Kanade,et al.  Coplanar Shadowgrams for Acquiring Visual Hulls of Intricate Objects , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Hermann von Helmholtz,et al.  Treatise on Physiological Optics , 1962 .

[14]  Ren Ng Fourier Slice Photography , 2005 .

[15]  Daphna Weinshall,et al.  Mosaicing New Views: The Crossed-Slits Projection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[18]  Gregory Dudek,et al.  Space occupancy using multiple shadowimages , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[19]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[20]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[21]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[22]  Ramesh Raskar,et al.  Characterizing the shadow space of camera-light pairs , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  H. Helmholtz Helmholtz's Treatise on Physiological Optics , 1963 .

[24]  Douglas Lanman,et al.  Shield fields: modeling and capturing 3D occluders , 2008, SIGGRAPH 2008.

[25]  Kenneth A. Loparo,et al.  Reconstruction of three-dimensional surfaces from two-dimensional binary images , 1989, IEEE Trans. Robotics Autom..

[26]  Harry Shum,et al.  Image-based rendering , 2006, Found. Trends Comput. Graph. Vis..

[27]  Tsuhan Chen,et al.  A Self-Reconfigurable Camera Array , 2004, Rendering Techniques.

[28]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[29]  Andrew Gardner,et al.  Capturing and Rendering with Incident Light Fields , 2003, Rendering Techniques.