PCA-coded aperture for light field photography

A light field, which is often understood as a set of dense multi-view images, has been utilized in various 2D/3D applications. Efficient light field acquisition using a coded aperture camera is the target problem considered in this paper. Specifically, the entire light field, which consists of many images, should be reconstructed from only a few images that are captured through different aperture patterns. In previous work, this problem has often been discussed from the context of compressed sensing (CS). In contrast, we formulated this problem from the perspective of principal component analysis (PCA) to derive optimal non-negative aperture patterns and a straight-forward reconstruction algorithm. Even though it is based on a conventional technique, our method has proven to be more accurate and much faster than a state-of-the-art CS-based method.

[1]  F. Okano,et al.  Gradient-index lens-array method based on real-time integral photography for three-dimensional images. , 1998, Applied optics.

[2]  Gordon Wetzstein,et al.  The light field stereoscope , 2015, ACM Trans. Graph..

[3]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Byoungho Lee,et al.  Additive light field displays , 2016, ACM Trans. Graph..

[5]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[6]  Ashok Veeraraghavan,et al.  Towards Motion Aware Light Field Video for Dynamic Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Gordon Wetzstein,et al.  Tensor displays , 2012, ACM Trans. Graph..

[8]  Pierre Vandergheynst,et al.  Light field compressive sensing in camera arrays , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[10]  Alexei A. Efros,et al.  Depth Estimation with Occlusion Modeling Using Light-Field Cameras , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[12]  Toshiaki Fujii,et al.  Multipoint Measuring System for Video and Sound - 100-camera and microphone system , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[13]  Mark A. Neifeld,et al.  Compressive light field imaging , 2010, Defense + Commercial Sensing.

[14]  Shree K. Nayar,et al.  Programmable Aperture Camera Using LCoS , 2012, IPSJ Trans. Comput. Vis. Appl..

[15]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[16]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[17]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[18]  Aggelos K. Katsaggelos,et al.  Compressive Light Field Sensing , 2012, IEEE Transactions on Image Processing.

[19]  Toshiaki Fujii,et al.  Displaying Real-World Light Fields With Stacked Multiplicative Layers: Requirement and Data Conversion for Input Multiview Images , 2016, Journal of Display Technology.

[20]  Tom E. Bishop,et al.  Light field superresolution , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[21]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[22]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[23]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[24]  P. Hanrahan,et al.  Digital light field photography , 2006 .

[25]  Saleem Zaroubi,et al.  2012 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP) , 2012, IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  Frédo Durand,et al.  Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain , 2014, ACM Trans. Graph..

[27]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mark A Neifeld,et al.  Feature-specific imaging. , 2003, Applied optics.

[29]  Toshiaki Fujii,et al.  Weighted 4D-DCT Basis for Compressively Sampled Light Fields , 2016, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[30]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.