3D color charts for camera spectral sensitivity estimation

Estimating spectral data such as camera sensor responses or illuminant spectral power distribution from raw RGB camera outputs is crucial in many computer vision applications. Usually, 2D color charts with various patches of known spectral reflectance are used as reference for such purpose. Deducing n-D spectral data (n»3) from 3D RGB inputs is an ill-posed problem that requires a high number of inputs. Unfortunately, most of the natural color surfaces have spectral reflectances that are well described by low-dimensional linear models, i.e. each spectral reflectance can be approximated by a weighted sum of the others. It has been shown that adding patches to color charts does not help in practice, because the information they add is redundant with the information provided by the first set of patches. In this paper, we propose to use spectral data of higher dimensionality by using 3D color charts that create inter-reflections between the surfaces. These inter-reflections produce multiplications between natural spectral curves and so provide non-linear spectral curves. We show that such data provide enough information for accurate spectral data estimation.

[1]  Thomas Mauthner,et al.  In defense of color-based model-free tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Mark S. Drew,et al.  Color constancy from mutual reflection , 1991, International Journal of Computer Vision.

[3]  Joost van de Weijer,et al.  Towards multispectral data acquisition with hand-held devices , 2013, 2013 IEEE International Conference on Image Processing.

[4]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[5]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Takeo Kanade,et al.  Shape from interreflections , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[7]  Graham Finlayson,et al.  Rank-based camera spectral sensitivity estimation. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[8]  Andrew Gilbert,et al.  Tracking Objects Across Cameras by Incrementally Learning Inter-camera Colour Calibration and Patterns of Activity , 2006, ECCV.

[9]  Michael S. Brown,et al.  Do It Yourself Hyperspectral Imaging with Everyday Digital Cameras , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Shu Wang,et al.  Multispectral Deep Neural Networks for Pedestrian Detection , 2016, BMVC.

[11]  Karim Helwani,et al.  Low complexity unsupervised multi-camera color calibration with application to panoramic video capturing , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[12]  Maryam Mohammadzadeh Darrodi,et al.  Reference data set for camera spectral sensitivity estimation. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[13]  H. Jensen Realistic Image Synthesis Using Photon Mapping , 2001 .

[14]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[15]  M. Hébert,et al.  Mutual Illumination and Color Constancy , 2017 .

[16]  Greg Welch,et al.  Ensuring color consistency across multiple cameras , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  R. Willoughby Solutions of Ill-Posed Problems (A. N. Tikhonov and V. Y. Arsenin) , 1979 .

[18]  Mark S. Drew,et al.  Separating a Color Signal into Illumination and Surface Reflectance Components: Theory and Applications , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Mark S. Drew,et al.  Calculating surface reflectance using a single-bounce model of mutual reflection , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[20]  B A Wandell,et al.  Linear models of surface and illuminant spectra. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[21]  Yoichi Sato,et al.  Interreflection Removal Using Fluorescence , 2014, ECCV.

[22]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[23]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[24]  Miao Liao,et al.  Interreflection removal for photometric stereo by using spectrum-dependent albedo , 2011, CVPR 2011.

[25]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[26]  Mark S. Drew,et al.  Color Space Analysis of Mutual Illumination , 1992, IEEE Trans. Pattern Anal. Mach. Intell..