Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

Sparsity in the Fourier domain is an important property that enables the dense reconstruction of signals, such as 4D light fields, from a small set of samples. The sparsity of natural spectra is often derived from continuous arguments, but reconstruction algorithms typically work in the discrete Fourier domain. These algorithms usually assume that sparsity derived from continuous principles will hold under discrete sampling. This article makes the critical observation that sparsity is much greater in the continuous Fourier spectrum than in the discrete spectrum. This difference is caused by a windowing effect. When we sample a signal over a finite window, we convolve its spectrum by an infinite sinc, which destroys much of the sparsity that was in the continuous domain. Based on this observation, we propose an approach to reconstruction that optimizes for sparsity in the continuous Fourier spectrum. We describe the theory behind our approach and discuss how it can be used to reduce sampling requirements and improve reconstruction quality. Finally, we demonstrate the power of our approach by showing how it can be applied to the task of recovering non-Lambertian light fields from a small number of 1D viewpoint trajectories.

[1]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Frédo Durand,et al.  Motion-invariant photography , 2008, SIGGRAPH 2008.

[3]  Piotr Indyk,et al.  Sample-optimal average-case sparse Fourier Transform in two dimensions , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[4]  Frédo Durand,et al.  Linear view synthesis using a dimensionality gap light field prior , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[6]  W. Freeman,et al.  Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections , 2008, ECCV.

[7]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[8]  Frédo Durand,et al.  A frequency analysis of light transport , 2005, SIGGRAPH '05.

[9]  M. Levoy,et al.  Wigner distributions and how they relate to the light field , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[10]  Frédo Durand,et al.  Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain , 2014, ACM Trans. Graph..

[11]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[12]  P MitchellDon Spectrally optimal sampling for distribution ray tracing , 1991 .

[13]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[14]  Frédo Durand,et al.  Frequency analysis and sheared reconstruction for rendering motion blur , 2009, ACM Trans. Graph..

[15]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[16]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[17]  Frédo Durand,et al.  4D frequency analysis of computational cameras for depth of field extension , 2009, SIGGRAPH '09.

[18]  David Salesin,et al.  Spatio-angular resolution tradeoffs in integral photography , 2006, EGSR '06.

[19]  Don P. Mitchell,et al.  Spectrally optimal sampling for distribution ray tracing , 1991, SIGGRAPH.

[20]  Anna C. Gilbert,et al.  Improved time bounds for near-optimal sparse Fourier representations , 2005, SPIE Optics + Photonics.

[21]  Tom E. Bishop,et al.  Light field superresolution , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[22]  Piotr Indyk,et al.  Simple and practical algorithm for sparse Fourier transform , 2012, SODA.

[23]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[24]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[25]  Piotr Indyk,et al.  Nearly optimal sparse fourier transform , 2012, STOC '12.

[26]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[27]  Paul S. Heckbert,et al.  Fundamentals of Texture Mapping and Image Warping , 1989 .

[28]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[29]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[30]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[31]  Frédo Durand,et al.  Unstructured Light Fields , 2012, Comput. Graph. Forum.

[32]  François X. Sillion,et al.  Fast calculation of soft shadow textures using convolution , 1998, SIGGRAPH.

[33]  Ren Ng Fourier slice photography , 2005, ACM Trans. Graph..

[34]  J. Goodman Introduction to Fourier optics , 1969 .

[35]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.