External Mask Based Depth and Light Field Camera

We present a method to convert a digital single-lens-reflex (DSLR) camera into a high resolution consumer depth and light field camera by affixing an external aperture mask to the main lens. Compared to the existing consumer depth and light field cameras, our camera is easy to construct with minimal additional costs and our design is camera and lens agnostic. The main advantage of our design is the ease of switching between an SLR camera and a native resolution depth/light field camera. Using an external mask is an important advantage over current light field camera designs since we do not need to modify the internals of the camera or the lens. Our camera sequentially acquires the angular components of the light field of a static scene by changing the location of the aperture in the mask. A consequence of our design is that the external aperture causes heavy vignetting in the acquired images. We calibrate the mask parameters and estimate multi-view scene depth under vignetting. In addition to depth, we show light field applications such as refocusing and defocus blur at the sensor resolution.

[1]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[3]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[4]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[5]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[7]  Radu Horaud,et al.  Time-of-Flight Cameras: Principles, Methods and Applications , 2012 .

[8]  Richard Szeliski,et al.  Extracting View-Dependent Depth Maps from a Collection of Images , 2004, International Journal of Computer Vision.

[9]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[10]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[11]  Shree K. Nayar,et al.  Ieee Transactions on Image Processing Computational Cameras: Convergence of Optics and Processing , 2022 .

[12]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[13]  Yael Pritch,et al.  Scene reconstruction from high spatio-angular resolution light fields , 2013, ACM Trans. Graph..

[14]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[15]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[17]  Gordon Wetzstein,et al.  Computational Plenoptic Imaging , 2011, SIGGRAPH '12.

[18]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[19]  Zhan Yu,et al.  An analysis of color demosaicing in plenoptic cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[21]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  H AdelsonEdward,et al.  Single Lens Stereo with a Plenoptic Camera , 1992 .

[23]  David Salesin,et al.  Spatio-angular resolution tradeoffs in integral photography , 2006, EGSR '06.