Multimode plenoptic imaging

The plenoptic function was originally defined as a complete record of the 3D structure of radiance in a scene and its dependence on a number of different parameters including position, angle, wavelength, polarization, etc. Recently-developed plenoptic cameras typically capture only the geometric aspects of the plenoptic function. Using this information, computational photography can render images with an infinite variety of features such as focus, depth of field, and parallax. Less attention has been paid to other, nonspatial, parameters of the plenoptic function that could also be captured. In this paper, we develop the microlens-based image sensor (aka the Lippmann sensor) as a generalized plenoptic capture device, able to capture additional information based on filters/modifiers placed on different microlenses. Multimodal capture can comprise many different parameters such as high-dynamic range, multispectral, and so on. For this paper we explore two particular examples in detail: polarization capture based on interleaved polarization filters, and capture with extended depth of field based on microlenses with different focal lengths.

[1]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[2]  G. Lippmann.  Epreuves reversibles donnant la sensation du relief , 1908 .

[3]  Narendra Ahuja,et al.  Split Aperture Imaging for High Dynamic Range , 2004, International Journal of Computer Vision.

[4]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[5]  Andrew Lumsdaine,et al.  Reducing Plenoptic Camera Artifacts , 2010, Comput. Graph. Forum.

[6]  Shree K. Nayar,et al.  Generalized Mosaicing: Wide Field of View Multispectral Imaging , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Sergio Goma,et al.  High Dynamic Range Image Capture with Plenoptic 2.0 Camera , 2009 .

[8]  E. Nicodemus.  Self-study manual on optical radiation measurements , 1976 .

[9]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[10]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[11]  S. Sternberg,et al.  Symplectic Techniques in Physics , 1984 .

[12]  Kurt Bernardo Wolf,et al.  Geometric Optics on Phase Space , 2004 .

[13]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[14]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[15]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[16]  A. Thetford.  Introduction to Matrix Methods in Optics , 1976 .

[17]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[18]  Shree K. Nayar,et al.  Generalized mosaicing: polarization panorama , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ravindra Athale,et al.  Flexible multimodal camera using a light field architecture , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[20]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Ravindra A. Athale,et al.  Modified light field architecture for reconfigurable multimode imaging , 2009, Optical Engineering + Applications.

[22]  G. Lippmann,et al.  Epreuves reversibles. Photographies integrals , 1908 .

[23]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[24]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[25]  Andrew Lumsdaine,et al.  Depth of Field in Plenoptic Cameras , 2009, Eurographics.

[26]  Andrew Lumsdaine,et al.  Rich image capture with plenoptic cameras , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[27]  Andrew Lumsdaine,et al.  Using Focused Plenoptic Cameras for Rich Image Capture , 2011, IEEE Computer Graphics and Applications.

[28]  Andrew Lumsdaine,et al.  The Multi-Focus Plenoptic Camera , 2011 .

[29]  Lennart Wietzke,et al.  Single lens 3D-camera with extended depth-of-field , 2012, Electronic Imaging.

[30]  Shree K. Nayar,et al.  Generalized Mosaicing : High Dynamic Range in a Wide Field of View 247 , 2001 .

[31]  Viktor Gruev,et al.  Fabrication of micro-polarizer array with polymer thin film , 2006 .

[32]  Shree K. Nayar,et al.  Enhancing resolution along multiple imaging dimensions using assorted pixels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.