Occlusion-Resistant Camera Design for Acquiring Active Environments

In this article, we present a detailed theoretical analysis and a prototype implementation of a family of cameras designed with the explicit goal of detecting and removing interfering dynamic occluders in real time, during live capture, as opposed to fixing the resulting artifacts a posteriori. Such an early-acquisition approach improves efficiency: more valid samples are acquired faster without worrying about moving occluders. One option for designing a camera to be unaffected by moving occluders is to sample through the occluder, but true x-ray-like vision is technically impractical. Another option is to sample around the occluder using a camera with a large effective aperture, but such an approach requires a bulky acquisition device. Yet another possibility of sampling around an occluder is to rely on second and higher order reflected rays that indirectly sample surfaces not directly visible. However, devising an acquisition device sufficiently sensitive and efficient to capture large environments using reflected rays will remain challenging for the foreseeable future.

[1]  Voicu Popescu,et al.  Lag Camera: A Moving Multi-Camera Array for Scence-Acquisition , 2006, J. Virtual Real. Broadcast..

[2]  Yiannis Aloimonos,et al.  A hierarchy of cameras for 3D photography , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[3]  Franc¸ois Blais,et al.  Review of 20 years of range sensor development , 2003, IS&T/SPIE Electronic Imaging.

[4]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[5]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[6]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[7]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[8]  Voicu Popescu,et al.  The depth discontinuity occlusion camera , 2006, I3D '06.

[9]  Frédo Durand,et al.  Defocus video matting , 2005, ACM Trans. Graph..

[10]  François Blais Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[11]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[12]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Li Zhang,et al.  Projection defocus analysis for scene capture and image display , 2006, ACM Trans. Graph..

[14]  Yiannis Aloimonos,et al.  A hierarchy of cameras for 3D photography , 2004, Comput. Vis. Image Underst..

[15]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[16]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..