One Shot 360-Degree Light Field Capture and Reconstruction with Depth Extraction Based on Optical Flow for Light Field Camera

A system for capturing 360-degree light field information of real-existing object in one shot and then optically reconstructing is proposed. A new depth extraction algorithm for light field cameras is proposed and various camera specifications for practical use of the algorithm are employed for the analysis. With a depth extraction method that was based on optical flow for light field camera, the depth information is extracted more accurately, according to the various specifications of light field camera. For 360-degree shooting, a simple capturing system composed of two mirrors and a light field camera is used. The capturing system has an advantage of being compact and inexpensive. The locations and orientations of the two mirrors are analyzed to optimize 360-degree light field recording. Holographic display is used to optically reconstruct the captured light field information. Experimental and simulation results are presented to support the proposed system and analysis.

[1]  Bahram Javidi,et al.  A 3D integral imaging optical see-through head-mounted display. , 2014, Optics express.

[2]  Byoungho Lee,et al.  Real-time depth controllable integral imaging pickup and reconstruction method with a light field camera. , 2015, Applied optics.

[3]  Hideo Saito,et al.  Light field rendering with omni-directional camera , 2003, Visual Communications and Image Processing.

[4]  Bahram Javidi,et al.  3D Integral Imaging Reconstruction of Occluded Objects Using Independent Component Analysis-Based K-Means Clustering , 2010, Journal of Display Technology.

[5]  Tom E. Bishop,et al.  The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Gilbae Park,et al.  Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging. , 2010, Optics express.

[7]  G Passalis,et al.  Enhanced reconstruction of three-dimensional shape and texture from integral photography images. , 2007, Applied optics.

[8]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  E. Boyden,et al.  Simultaneous whole-animal 3D-imaging of neuronal activity using light-field microscopy , 2014, Nature Methods.

[10]  Jason Geng,et al.  Structured-light 3D surface imaging: a tutorial , 2011 .

[11]  Joonku Hahn,et al.  Holographic head-mounted display with RGB light emitting diode light source. , 2014, Optics express.

[12]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[13]  M. Levoy,et al.  Light field microscopy , 2006, SIGGRAPH 2006.

[14]  Vladan Velisavljevic,et al.  Refocusing distance of a standard plenoptic camera. , 2016, Optics express.

[15]  Yasuhiro Takaki,et al.  Super multi-view windshield display for long-distance image information presentation. , 2011, Optics express.

[16]  Bahram Javidi,et al.  Formation of real, orthoscopic integral images by smart pixel mapping. , 2005, Optics express.

[17]  Jae-Hyeung Park,et al.  Hologram synthesis of three-dimensional real objects using portable integral imaging camera. , 2013, Optics express.

[18]  Joonku Hahn,et al.  Anamorphic optical transformation of an amplitude spatial light modulator to a complex spatial light modulator with square pixels [invited]. , 2014, Applied optics.

[19]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[20]  Gordon Wetzstein,et al.  The light field stereoscope , 2015, ACM Trans. Graph..

[21]  Byoungho Lee,et al.  Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography. , 2008, Applied optics.

[22]  Byoungho Lee,et al.  Real-time integral imaging system for light field microscopy. , 2014, Optics express.

[23]  Tieniu Tan,et al.  High quality depth map estimation of object surface from light-field images , 2017, Neurocomputing.

[24]  Gordon Wetzstein,et al.  Light field optical flow for refractive surface reconstruction , 2012, Other Conferences.

[25]  Ruigang Yang,et al.  Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.