Diffraction model of a plenoptic camera for in-situ space exploration

This paper provides a detailed system theoretical model of a plenoptic camera with the aim to provide in-depth understanding of the plenoptic data recording concept and its effects. Plenoptic cameras, also known as light field cameras, were firstly thought of in the beginning of the 20th century and became recently possible thanks to rapid development of processing hardware and the increase of camera sensor resolution. Despite being a new type of sensor, they are operated in the same way as conventional cameras, but offer several advantages. A plenoptic camera consists of a main lens and a lenslet array (microlens array) right in front of the detector. The microlens array causes not only the recording of the incident location of a light ray on the sensor, as it is done by a conventional camera, but also the incident direction. Such a record can be represented by a 4-D data set known as the light field. In fact, by inserting a microlens array any conventional camera can be transformed into a plenoptic camera. The plenoptic recording concept and the 4-D light field provide multiple advantages over conventional cameras. For example, a single recorded light field allows first, to reconstruct novel views with small changes in viewpoint, second, to create a depth map, and third, to refocus images after the data capture. Hence, the process of focusing is shifted from hardware to software. Last, but not least, plenoptic cameras allow an extended depth of field in comparison to a conventional camera and the use of a bigger camera aperture. Most of the mentioned advantages become particularly effective at close-range to an object. The German Aerospace Center performs research on plenoptic cameras for close-range imaging in space. Possible applications are for example robot vision with plenoptic cameras for robotic arm operations during on-orbit servicing missions or the use of plenoptic cameras on rovers in the course of exploration missions to other planets. Those application scenarios and the demanding conditions in space require thorough comprehension of plenoptic cameras. For this purpose, this paper shall provide a detailed model of plenoptic cameras, which allows to derive camera parameters and optimize them with particular attention to the user requirements and to generate synthetic data. The latter can be utilized to assess the evaluation algorithms, which are not mentioned in detail in this paper. The modeling of the plenoptic camera is mainly based on the theory of geometric optics expanded by elements of diffraction optics.

[1]  Qionghai Dai,et al.  Point spread function and depth-invariant focal sweep point spread function for plenoptic camera 2.0. , 2017, Optics express.

[2]  Kenneth S. Edgett,et al.  MAHLI on Mars: lessons learned operating a geoscience camera on a landed payload robotic arm , 2016 .

[3]  Lennart Wietzke,et al.  Single lens 3D-camera with extended depth-of-field , 2012, Electronic Imaging.

[4]  Mark T. Lemmon,et al.  The Mars Science Laboratory (MSL) Mars Hand Lens Imager (MAHLI) Flight Instrument , 2009 .

[5]  F. Tony Ghaemi,et al.  Design and fabrication of lenses for the color science cameras aboard the Mars Science Laboratory rover , 2009 .

[6]  Eric Holk,et al.  Depth map estimation for plenoptic images , 2011 .

[7]  Massimo Turola.  Investigation of plenoptic imaging systems: a wave optics approach , 2016 .

[8]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[9]  Andrew Lumsdaine,et al.  The multifocus plenoptic camera , 2011, Electronic Imaging.

[10]  G. Lippmann.  Epreuves reversibles donnant la sensation du relief , 1908 .

[11]  Andrew Lumsdaine,et al.  Depth of Field in Plenoptic Cameras , 2009, Eurographics.

[12]  Jürgen Jahns.  Photonik: Grundlagen, Komponenten und Systeme , 2000 .

[13]  Richard Szeliski,et al.  Refocusing plenoptic images using depth-adaptive splatting , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[14]  Ralf Reulke,et al.  Systemtheoretische Grundlagen optoelektronischer Sensoren. , 1995 .

[15]  Daniel R. DiBiase,et al.  Mars hand lens imager: Lens mechanical design , 2009, 2009 IEEE Aerospace conference.

[16]  Ulrike Talbiersky,et al.  A new method to validate the usage of Fresnel approximation instead of Kirchhoff diffraction formula for calculations concerning camera systems , 2010, Photonics Europe.

[17]  Gordon Wetzstein,et al.  Computational Plenoptic Imaging , 2011, SIGGRAPH '12.

[18]  Ulrike Krutz.  Analytische Betrachtung optisch abbildender Sensorsysteme im Rahmen der AsteroidFinder-Mission , 2011 .

[19]  Edmund Y Lam,et al.  Computational photography with plenoptic camera and light field capture: tutorial. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  Zeev Zalevsky,et al.  Computation considerations and fast algorithms for calculating the diffraction integral , 1997 .

[21]  Andrew Lumsdaine,et al.  Focused plenoptic camera and rendering , 2010, J. Electronic Imaging.

[22]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[23]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[24]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .