Homogeneous codes for energy-efficient illumination and imaging

Programmable coding of light between a source and a sensor has led to several important results in computational illumination, imaging and display. Little is known, however, about how to utilize energy most effectively, especially for applications in live imaging. In this paper, we derive a novel framework to maximize energy efficiency by "homogeneous matrix factorization" that respects the physical constraints of many coding mechanisms (DMDs/LCDs, lasers, etc.). We demonstrate energy-efficient imaging using two prototypes based on DMD and laser illumination. For our DMD-based prototype, we use fast local optimization to derive codes that yield brighter images with fewer artifacts in many transport probing tasks. Our second prototype uses a novel combination of a low-power laser projector and a rolling shutter camera. We use this prototype to demonstrate never-seen-before capabilities such as (1) capturing live structured-light video of very bright scenes---even a light bulb that has been turned on; (2) capturing epipolar-only and indirect-only live video with optimal energy efficiency; (3) using a low-power projector to reconstruct 3D objects in challenging conditions such as strong indirect light, strong ambient light, and smoke; and (4) recording live video from a projector's---rather than the camera's---point of view.

[1]  Mohit Gupta,et al.  MC3D: Motion Contrast 3D Scanning , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[2]  Shree K. Nayar,et al.  When Does Computational Imaging Improve Performance? , 2013, IEEE Transactions on Image Processing.

[3]  D Aspinall,et al.  Real-time multiplexing of dispersed spectra in any wavelength region. , 1968, Applied optics.

[4]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[5]  Gordon Wetzstein,et al.  Tensor displays , 2012, ACM Trans. Graph..

[6]  Shree K. Nayar,et al.  Structured Light in Sunlight , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Srinivasa G. Narasimhan,et al.  A low-power structured light sensor for outdoor scene reconstruction and dominant material identification , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Wolfgang Heidrich,et al.  Light reallocation for high contrast projection using an analog micromirror array , 2010, ACM Trans. Graph..

[9]  Jean Ponce,et al.  Convex Sparse Matrix Factorizations , 2008, ArXiv.

[10]  Matthew O'Toole,et al.  3D Shape and Indirect Appearance by Structured Light Transport , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Ashok Veeraraghavan,et al.  Can we beat Hadamard multiplexing? Data driven design and analysis for computational imaging systems , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[12]  R. Raskar,et al.  Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization , 2010, SIGGRAPH 2010.

[13]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[14]  Douglas Lanman.  Content-adaptive parallax barriers : optimizing dual-layer 3 D displays using low-rank light field factorization Citation , 2010 .

[15]  J. A. Decker,et al.  Experimental operation of a hadamard spectrometer. , 1969, Applied optics.

[16]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[17]  N KutulakosKiriakos,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015 .

[18]  Shree K. Nayar,et al.  Compressive Structured Light for Recovering Inhomogeneous Participating Media , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Shree K. Nayar,et al.  Multiplexing for Optimal Lighting , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Shree K. Nayar,et al.  Video from a single coded exposure photograph using a learned over-complete dictionary , 2011, 2011 International Conference on Computer Vision.

[21]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[22]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[23]  Ashok Veeraraghavan,et al.  A Framework for Analysis of Computational Imaging Systems: Role of Signal Prior, Sensor Noise and Multiplexing , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ramesh Raskar,et al.  Coded Strobing Photography: Compressive Sensing of High Speed Periodic Videos , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Wolfgang Heidrich,et al.  Efficient freeform lens optimization for computational caustic displays. , 2015, Optics express.

[26]  Srinivasa G. Narasimhan,et al.  Beyond Perspective Dual Photography With Illumination Masks , 2015, IEEE Transactions on Image Processing.

[27]  N. Sloane,et al.  Hadamard transform optics , 1979 .

[28]  Matthew O'Toole,et al.  Primal-dual coding to probe light transport , 2012, ACM Trans. Graph..

[29]  Shree K. Nayar,et al.  Programmable imaging using a digital micromirror array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[30]  Anders Ballestad,et al.  Efficient, high brightness, high dynamic range projection , 2014, SIGGRAPH '14.

[31]  René Vidal,et al.  Structured Low-Rank Matrix Factorization: Optimality, Algorithm, and Applications to Image Processing , 2014, ICML.