Do It Yourself Hyperspectral Imaging with Everyday Digital Cameras

Capturing hyperspectral images requires expensive and specialized hardware that is not readily accessible to most users. Digital cameras, on the other hand, are significantly cheaper in comparison and can be easily purchased and used. In this paper, we present a framework for reconstructing hyperspectral images by using multiple consumer-level digital cameras. Our approach works by exploiting the different spectral sensitivities of different camera sensors. In particular, due to the differences in spectral sensitivities of the cameras, different cameras yield different RGB measurements for the same spectral signal. We introduce an algorithm that is able to combine and convert these different RGB measurements into a single hyperspectral image for both indoor and outdoor scenes. This camera-based approach allows hyperspectral imaging at a fraction of the cost of most existing hyperspectral hardware. We validate the accuracy of our reconstruction against ground truth hyperspectral images (using both synthetic and real cases) and show its usage on relighting applications.

[1]  Chi-Wing Fu,et al.  Visualizing Multiwavelength Astrophysical Data , 2008, IEEE Transactions on Visualization and Computer Graphics.

[2]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[3]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[4]  Bruce J. Tromberg,et al.  Face Recognition in Hyperspectral Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  G. de Bruin,et al.  QUANTITATIVE HYPERSPECTRAL IMAGING OF HISTORICAL DOCUMENTS: TECHNIQUE AND APPLICATIONS , 2008 .

[6]  Brian V. Funt,et al.  A data set for color research , 2002 .

[7]  Katsushi Ikeuchi,et al.  Camera Spectral Sensitivity and White Balance Estimation from Sky Images , 2013, International Journal of Computer Vision.

[8]  Er-Wei Bai,et al.  Least squares solutions of bilinear equations , 2006, Syst. Control. Lett..

[9]  J. Parkkinen,et al.  Characteristic spectra of Munsell colors , 1989 .

[10]  Mona Lisa Spectral imaging of Leonardo Da Vinci ’ s , 2006 .

[11]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[12]  Stephen Lin,et al.  A New In-Camera Imaging Model for Color Computer Vision and Its Application , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Michael S. Brown,et al.  Training-Based Spectral Reconstruction from a Single RGB Image , 2014, ECCV.

[14]  W. R. Windham,et al.  CALIBRATION OF A PUSHBROOM HYPERSPECTRAL IMAGING SYSTEM FOR AGRICULTURAL INSPECTION , 2003 .

[15]  C. Tomasi,et al.  Systems of Bilinear Equations , 1997 .

[16]  Shree K. Nayar,et al.  Multispectral Imaging Using Multiplexed Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Steven D. Hordley,et al.  A multi-spectral image database and its application to image rendering across illumination , 2004, Third International Conference on Image and Graphics (ICIG'04).

[18]  Takahiro Okabe,et al.  Fast Spectral Reflectance Recovery Using DLP Projector , 2010, ACCV.

[19]  Sabine Süsstrunk,et al.  What is the space of spectral sensitivity functions for digital color cameras? , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[20]  Stephen Lin,et al.  A Prism-Mask System for Multispectral Video Acquisition. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[21]  Peter Morovic,et al.  Metamer-set-based approach to estimating surface reflectance from camera RGB. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  L. Maloney Evaluation of linear models of surface spectral reflectance with small numbers of parameters. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[23]  S. H. Amirshahi,et al.  Reconstruction of reflectance data using an interpolation technique. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Michael S. Brown,et al.  Visual enhancement of old documents with hyperspectral imaging , 2011, Pattern Recognit..

[25]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Mohamed Saber Naceur,et al.  A Spectral and Spatial Source Separation of Multispectral Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[28]  Moshe Ben-Ezra,et al.  Multi-Spectral Imaging by Optimized Wide Band Illumination , 2008, International Journal of Computer Vision.

[29]  David Casasent,et al.  Fast feature selection algorithm for poultry skin tumor detection in hyperspectral data , 2009 .

[30]  Glenn Healey,et al.  What is the spectral dimensionality of illumination functions in outdoor scenes? , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[31]  Christophe Collet,et al.  Markov model for multispectral image analysis: application to small magellanic cloud segmentation , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[32]  D. B. Judd,et al.  Spectral Distribution of Typical Daylight as a Function of Correlated Color Temperature , 1964 .

[33]  Nahum Gat,et al.  Imaging spectroscopy using tunable filters: a review , 2000, SPIE Defense + Commercial Sensing.

[34]  Renfu Lu,et al.  Detection of bruises on apples using near-infrared hyperspectral imaging , 2003 .

[35]  J. Cohen Dependency of the spectral reflectance curves of the Munsell color chips , 1964 .