Efficient spectral reconstruction using a trichromatic camera via sample optimization

Training-based multispectral reconstruction can effectively recover spectral reflectance of captured objects using a trichromatic camera. However, existing methods are based on synthesized data, and the sizes of training sample set (e.g., multispectral images, reflectance targets) are usually large. In this paper, we present a spectral reconstruction approach using real measured data. To improve the efficiency and accuracy of spectral reconstruction, we propose a volume maximization method for sample optimization without any prior knowledge of light and cameras. We use heuristic global search algorithms to optimize samples and give an efficient spectral reconstruction method which is suitable for sparse sampling. Experimental results show that the proposed sample selection method outperforms other existing methods in terms of both spectral and colorimetric reconstruction errors. Moreover, the proposed reconstruction method achieves higher efficiency and accuracy due to lower sample redundancy.

[1]  J. Parkkinen,et al.  Characteristic spectra of Munsell colors , 1989 .

[2]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[3]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[4]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[5]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[6]  Roy S. Berns,et al.  Spectral Imaging Target Development Based on Hiearchical Cluster Analysis , 2004, Color Imaging Conference.

[7]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[8]  Stephen Westland,et al.  Methods for optimal color selection , 2006 .

[9]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[10]  Shree K. Nayar,et al.  Multispectral Imaging Using Multiplexed Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Ville Heikkinen,et al.  Regularized learning framework in the estimation of reflectance spectra from camera responses. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  J. Hardeberg,et al.  Representation and estimation of spectral reflectances using projection on PCA and wavelet bases , 2008 .

[13]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[14]  Hui-Liang Shen,et al.  Optimal selection of representative colors for spectral reflectance reconstruction in a multispectral imaging system. , 2008, Applied optics.

[15]  S. H. Amirshahi,et al.  Reconstruction of reflectance spectra using weighted principal component analysis , 2008 .

[16]  Malik Magdon-Ismail,et al.  On selecting a maximum volume sub-matrix of a matrix and related problems , 2009, Theor. Comput. Sci..

[17]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[18]  Takahiro Okabe,et al.  Fast Spectral Reflectance Recovery Using DLP Projector , 2010, ACCV.

[19]  H. Seidel,et al.  Pattern-aware Deformation Using Sliding Dockers , 2011, SIGGRAPH 2011.

[20]  Sabine Süsstrunk,et al.  Multi-spectral SIFT for scene category recognition , 2011, CVPR 2011.

[21]  Maneesh Agrawala,et al.  Illumination decomposition for material recoloring with consistent interreflections , 2011, SIGGRAPH 2011.

[22]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Gongguo Tang,et al.  Estimation of reflectance from camera responses by the regularized local linear model. , 2011, Optics letters.

[24]  Jong-Il Park,et al.  Fast model-based multispectral imaging using nonnegative principal component analysis. , 2012, Optics letters.

[25]  Stephen Lin,et al.  Interactive chromaticity mapping for multispectral images , 2013, The Visual Computer.

[26]  Xin Du,et al.  Channel selection for multispectral color imaging using binary differential evolution. , 2014, Applied optics.

[27]  Michael S. Brown,et al.  Training-Based Spectral Reconstruction from a Single RGB Image , 2014, ECCV.

[28]  Guoyan Zheng,et al.  Comparison study of Gauss, MQ and TPS for interpolation application , 2014 .

[29]  Steve Marschner,et al.  Matching Real Fabrics with Micro-Appearance Models , 2015, ACM Trans. Graph..

[30]  Dongming Lu,et al.  Content‐Independent Multi‐Spectral Display Using Superimposed Projections , 2015, Comput. Graph. Forum.

[31]  Michael S. Brown,et al.  Do It Yourself Hyperspectral Imaging with Everyday Digital Cameras , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Timo Kunkel,et al.  Practical Low‐Cost Recovery of Spectral Power Distributions , 2016, Comput. Graph. Forum.

[33]  Yoichi Sato,et al.  Exploiting Spectral-Spatial Correlation for Coded Hyperspectral Image Restoration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Dawei Zhang,et al.  A method for selecting training samples based on camera response , 2016 .

[35]  Ville Heikkinen,et al.  Spectral imaging using consumer-level devices and kernel-based regression. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[36]  Boaz Arad,et al.  Sparse Recovery of Hyperspectral Signal from Natural RGB Images , 2016, ECCV.

[37]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[38]  Ravi Ramamoorthi,et al.  Deep high dynamic range imaging of dynamic scenes , 2017, ACM Trans. Graph..

[39]  Dimitris Samaras,et al.  EyeOpener: Editing Eyes in the Wild , 2017, ACM Trans. Graph..

[40]  Minh N. Do,et al.  DASC: Robust Dense Descriptor for Multi-Modal and Multi-Spectral Correspondence Estimation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Hans-Peter Seidel,et al.  Practical Capture and Reproduction of Phosphorescent Appearance , 2017, Comput. Graph. Forum.