Reconstruction of compressively sampled light fields using a weighted 4D-DCT basis

The coded aperture/mask technique enables us to capture light field data in a compressive way through a single camera. A pixel value recorded by such a camera is a summation of the light rays that pass though different positions on the coded aperture/mask. The target light field can be reconstructed from the recorded pixel values by using prior information of the light field signal. As prior information, a dictionary (light field atoms), which was learned from training datasets, was used in the current state of the art. Meanwhile, it was reported that general bases such as DCT were not suitable to efficiently represent prior information. In this work, however, we demonstrate that a 4D-DCT basis works surprisingly better if it is combined with a weighting scheme in which the amplitude difference in DCT coefficients is considered. Simulation results using 18 light field datasets are reported to show the superior performance of the weighted 4D-DCT basis to the learned dictionary.

[1]  Aggelos K. Katsaggelos,et al.  Compressive Light Field Sensing , 2012, IEEE Transactions on Image Processing.

[2]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[3]  Pierre Vandergheynst,et al.  Light field compressive sensing in camera arrays , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Toshiaki Fujii,et al.  Reconstruction of Compressively Sampled Ray Space by Statistically Weighted Model , 2014, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[5]  Ashok Veeraraghavan,et al.  Towards Motion Aware Light Field Video for Dynamic Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[7]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[8]  F. Okano,et al.  Gradient-index lens-array method based on real-time integral photography for three-dimensional images. , 1998, Applied optics.

[9]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[10]  Shree K. Nayar,et al.  Programmable Aperture Camera Using LCoS , 2010, IPSJ Trans. Comput. Vis. Appl..

[11]  Toshiaki Fujii,et al.  Reconstruction of compressively sampled ray space by using DCT basis and statistically weighted L1 norm optimization , 2013, Electronic Imaging.

[12]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[13]  Toshiaki Fujii,et al.  Multipoint Measuring System for Video and Sound - 100-camera and microphone system , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[14]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[15]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[16]  David Salesin,et al.  Spatio-angular resolution tradeoffs in integral photography , 2006, EGSR '06.

[17]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[18]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[20]  Ulrich Muehlmann,et al.  A new high speed cmos camera for real-time tracking applications , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[21]  Takeshi Naemura,et al.  Design and Implementation of a Real-Time Video-Based Rendering System Using a Network Camera Array , 2009, IEICE Trans. Inf. Syst..

[22]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[23]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[24]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[25]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.