A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging

We propose a non-permanent add-on that enables plenoptic imaging with standard cameras. Our design is based on a physical copying mechanism that multiplies a sensor image into a number of identical copies that still carry the plenoptic information of interest. Via different optical filters, we can then recover the desired information. A minor modification of the design also allows for aperture sub-sampling and, hence, light-field imaging. As the filters in our design are exchangeable, a reconfiguration for different imaging purposes is possible. We show in a prototype setup that high dynamic range, multispectral, polarization, and light-field imaging can be achieved with our design.

[1]  Narendra Ahuja,et al.  Split Aperture Imaging for High Dynamic Range , 2004, International Journal of Computer Vision.

[2]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[3]  Shree K. Nayar,et al.  Generalized mosaicing: polarization panorama , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[5]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[6]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[7]  Ravindra Athale,et al.  Flexible multimodal camera using a light field architecture , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[8]  Gordon Wetzstein,et al.  Computational Plenoptic Imaging , 2011, SIGGRAPH '12.

[9]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Douglas Lanman,et al.  Shield fields: modeling and capturing 3D occluders , 2008, SIGGRAPH 2008.

[11]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[12]  Jun Tanida,et al.  Multispectral imaging system by compact compound optics , 2003, SPIE Optics + Photonics.

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[14]  Shree K. Nayar,et al.  Multispectral Imaging Using Multiplexed Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[16]  Andrew Lumsdaine,et al.  Using Focused Plenoptic Cameras for Rich Image Capture , 2011, IEEE Computer Graphics and Applications.

[17]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Hans-Peter Seidel,et al.  A kaleidoscopic approach to surround geometry and reflectance acquisition , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Bernard Delabre,et al.  First light of SINFONI at the VLT , 2004 .

[20]  Michael W. Kudenov,et al.  Compact snapshot real-time imaging spectrometer , 2011, Security + Defence.

[21]  Gordon Wetzstein,et al.  A theory of plenoptic multiplexing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Shree K. Nayar,et al.  Enhancing resolution along multiple imaging dimensions using assorted pixels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[24]  Marc Levoy,et al.  Synthetic aperture confocal imaging , 2004, SIGGRAPH 2004.

[25]  Shree K. Nayar,et al.  Instant dehazing of images using polarization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Stephen Lin,et al.  High resolution multispectral video capture with a hybrid camera system , 2011, CVPR 2011.

[27]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  M E Gehm,et al.  Single-shot compressive spectral imaging with a dual-disperser architecture. , 2007, Optics express.

[29]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Gábor Horváth,et al.  Imaging polarimetry of the circularly polarizing cuticle of scarab beetles (Coleoptera: Rutelidae, Cetoniidae) , 2006, Vision Research.

[31]  Ken Perlin,et al.  Measuring bidirectional texture reflectance with a kaleidoscope , 2003, ACM Trans. Graph..

[32]  Stephen Lin,et al.  A Prism-Mask System for Multispectral Video Acquisition. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[33]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[34]  Shree K. Nayar,et al.  Ieee Transactions on Image Processing Computational Cameras: Convergence of Optics and Processing , 2022 .

[35]  David Chenault,et al.  Four camera complete Stokes imaging polarimeter , 2008, SPIE Defense + Commercial Sensing.

[36]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[37]  Sven Wanner,et al.  Spatial and Angular Variational Super-Resolution of 4D Light Fields , 2012, ECCV.

[38]  I. Yamaguchi,et al.  Simultaneous acquisition of spectral image information. , 1991, Optics letters.

[39]  Shree K. Nayar,et al.  Optical Splitting Trees for High-Precision Monocular Imaging , 2007, IEEE Computer Graphics and Applications.

[40]  Steve Mann,et al.  ON BEING `UNDIGITAL' WITH DIGITAL CAMERAS: EXTENDING DYNAMIC RANGE BY COMBINING DIFFERENTLY EXPOSED PICTURES , 1995 .

[41]  M. Descour,et al.  Computed-tomography imaging spectrometer: experimental calibration and reconstruction results. , 1995, Applied optics.

[42]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[43]  Jussi Parkkinen,et al.  Vector-subspace model for color representation , 1990 .

[44]  Satoru Toyooka,et al.  Two-dimensional spectral analysis using broad-band filters , 1997 .

[45]  Hans-Peter Seidel,et al.  Three-dimensional kaleidoscopic imaging , 2011, CVPR 2011.

[46]  László Neumann,et al.  Applications of High Precision Imaging Polarimetry , 2008, CAe.

[47]  Michael Wimmer,et al.  Practical Spectral Photography , 2012, Comput. Graph. Forum.

[48]  Reinhard Klein,et al.  Spectralization: Reconstructing spectra from sparse data , 2010, Comput. Graph. Forum.

[49]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[50]  Ren Ng Fourier Slice Photography , 2005 .

[51]  Tomoyuki Nishita,et al.  Extracting depth and matte using a color-filtered aperture , 2008, SIGGRAPH Asia '08.

[52]  Robert L Cook,et al.  Wavelet noise , 2005, ACM Trans. Graph..