Exploiting parallax in panoramic capture to construct light fields

We show that the parallax motion resulting from non-nodal rotation in panorama capture can be exploited for light field construction from commodity hardware. Automated panoramic image capture typically seeks to rotate a camera exactly about its nodal point, for which no parallax motion is observed. This can be difficult or impossible to achieve due to limitations of the mounting or optical systems, and consequently a wide range of captured panoramas suffer from parallax between images. We show that by capturing such imagery over a regular grid of camera poses, then appropriately transforming the captured imagery to a common parameterisation, a light field can be constructed. The resulting four-dimensional image encodes scene geometry as well as texture, allowing an increasingly rich range of light field processing techniques to be applied. Employing an Ocular Robotics REV25 camera pointing system, we demonstrate light field capture, refocusing and low-light image enhancement.

[1]  Leonard T. Bruton,et al.  A 4D frequency-planar IIR filter and its application to light field processing , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[2]  Hailin Jin,et al.  Light field video stabilization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Stefan B. Williams,et al.  Light field image denoising using a linear 4D frequency-hyperfan all-in-focus filter , 2013, Electronic Imaging.

[4]  Sven Wanner,et al.  Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Tom E. Bishop,et al.  Full-Resolution Depth Map Estimation from an Aliased Plenoptic Light Field , 2010, ACCV.

[7]  Richard Szeliski,et al.  Refocusing plenoptic images using depth-adaptive splatting , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[8]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[9]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[10]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[11]  Stefan B. Williams,et al.  Seabed modeling and distractor extraction for mobile AUVs using light field filtering , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .