Illuminant estimation in multispectral imaging.

With the advancement in sensor technology, the use of multispectral imaging is gaining wide popularity for computer vision applications. Multispectral imaging is used to achieve better discrimination between the radiance spectra, as compared to the color images. However, it is still sensitive to illumination changes. This study evaluates the potential evolution of illuminant estimation models from color to multispectral imaging. We first present a state of the art on computational color constancy and then extend a set of algorithms to use them in multispectral imaging. We investigate the influence of camera spectral sensitivities and the number of channels. Experiments are performed on simulations over hyperspectral data. The outcomes indicate that extension of computational color constancy algorithms from color to spectral gives promising results and may have the potential to lead towards efficient and stable representation across illuminants. However, this is highly dependent on spectral sensitivities and noise. We believe that the development of illuminant invariant multispectral imaging systems will be a key enabler for further use of this technology.

[1]  John K. Tsotsos,et al.  From [R, G, B] to Surface Reflectance: Computing Color Constant Descriptors in Images , 1987, IJCAI.

[2]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[3]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  David H Brainard,et al.  Surface color perception and equivalent illumination models. , 2011, Journal of vision.

[5]  J. Hernández-Andrés,et al.  Color and spectral analysis of daylight in southern Europe. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[6]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[7]  Xiaoou Tang,et al.  Deep Specialized Network for Illuminant Estimation , 2016, ECCV.

[8]  Hideaki Haneishi,et al.  CIE-XYZ fitting by multispectral images and mean square error minimization with a linear interpolation function , 2004 .

[9]  Yassine Ruichek,et al.  Energy balance in Spectral Filter Array camera design , 2017, Journal of the European Optical Society-Rapid Publications.

[10]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[11]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[12]  Sivalogeswaran Ratnasingam,et al.  Study of the photodetector characteristics of a camera for color constancy in natural scenes. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[13]  Joel Pokorny,et al.  Spectral sensitivities of the human cones , 2013 .

[14]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Guillermo Sapiro,et al.  Color and Illuminant Voting , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  E H Land,et al.  Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[17]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[19]  Brian V. Funt,et al.  Color constancy under varying illumination , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Kosuke Sato,et al.  An object recognition through continuous spectral images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[21]  Graham D. Finlayson,et al.  The Reproduction Angular Error for Evaluating the Performance of Illuminant Estimation Algorithms , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  W. Cohen,et al.  Hyperspectral versus multispectral data for estimating leaf area index in four different biomes , 2004 .

[23]  Brian V. Funt,et al.  Camera characterization for color research , 2002 .

[24]  Brian A. Wandell,et al.  The Synthesis and Analysis of Color Images , 1992, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Mehdi Rezagholizadeh,et al.  Image Sensor Modeling: Color Measurement at Low Light Levels , 2014 .

[26]  Ning Wang,et al.  Edge-Based Color Constancy via Support Vector Regression , 2009, IEICE Trans. Inf. Syst..

[27]  Mehdi Rezagholizadeh,et al.  Edge-Based and Efficient Chromaticity Spatio-spectral Models for Color Constancy , 2013, 2013 International Conference on Computer and Robot Vision.

[28]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Pierre Gouton,et al.  Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition , 2016, Sensors.

[30]  L. Maloney,et al.  Color constancy: a method for recovering surface spectral reflectance , 1987 .

[31]  Jean-Baptiste Thomas,et al.  Illuminant estimation from uncalibrated multispectral images , 2015, 2015 Colour and Visual Computing Symposium (CVCS).

[32]  Sven J. Dickinson,et al.  Object Representation and Recognition , 1999 .

[33]  Kinjiro Amano,et al.  Recovering spectral data from natural scenes with an RGB digital camera and colored filters , 2007 .

[34]  Marc Ebner,et al.  Color Constancy , 2007, Computer Vision, A Reference Guide.

[35]  In-So Kweon,et al.  Dichromatic-based color constancy using dichromatic slope and dichromatic line space , 2005, IEEE International Conference on Image Processing 2005.

[36]  Changhong Liu,et al.  Application of Multispectral Imaging to Determine Quality Attributes and Ripeness Stage in Strawberry Fruit , 2014, PloS one.

[37]  G. Finlayson,et al.  Chromagenic Colour Constancy , 2005 .

[38]  D H Brainard,et al.  Color constancy in the nearly natural image. I. Asymmetric matches. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[40]  Turgay Çelik,et al.  Adaptive colour constancy algorithm using discrete wavelet transform , 2012, Comput. Vis. Image Underst..

[41]  Seoung Wug Oh,et al.  Approaching the computational color constancy as a classification problem through deep learning , 2016, Pattern Recognit..

[42]  Kobus Barnard,et al.  Estimating the scene illumination chromaticity by using a neural network. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Javier Romero,et al.  Unsupervised illuminant estimation from natural scenes: an RGB digital camera suffices. , 2008, Applied optics.

[44]  Jon Yngve Hardeberg,et al.  Spatial arrangement of color filter array for multispectral image acquisition , 2011, Electronic Imaging.

[45]  M D'Zmura,et al.  Mechanisms of color constancy. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[46]  Stephen Westland,et al.  Recovering spectral information using digital camera systems , 2001 .

[47]  Pierre Gouton,et al.  Multispectral Filter Arrays: Recent Advances and Practical Implementation , 2014, Sensors.

[48]  Jon Yngve Hardeberg,et al.  Spectrogenic imaging: a novel approach to multispectral imaging in an uncontrolled environment. , 2014, Optics express.

[49]  Jing Wang,et al.  Robust automatic white balance algorithm using gray color points in images , 2006, IEEE Transactions on Consumer Electronics.

[50]  Brian V. Funt,et al.  Color Constancy for Scenes with Varying Illumination , 1997, Comput. Vis. Image Underst..

[51]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[52]  Vivek Agarwal,et al.  Machine learning approach to color constancy , 2007, Neural Networks.

[53]  Steven D. Hordley,et al.  Scene illuminant estimation: Past, present, and future , 2006 .

[54]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[55]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Graham D. Finlayson,et al.  The bright-chromagenic algorithm for illuminant estimation , 2007, Color Imaging Conference.

[57]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Lou,et al.  UvA-DARE (Digital Academic Repository) Color Constancy by Deep Learning Color Constancy by Deep Learning , 2015 .

[59]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[60]  Roy S. Berns,et al.  Spectral Estimation Using Trichromatic Digital Cameras , 1999 .

[61]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Brian V. Funt,et al.  Multispectral color constancy: real image tests , 2007, Electronic Imaging.

[63]  D. Foster,et al.  Frequency of metamerism in natural scenes , 2006 .

[64]  Jon Yngve Hardeberg,et al.  Multispectral imaging: narrow or wide band filters? , 2014 .

[65]  Raimondo Schettini,et al.  Consensus-based framework for illuminant chromaticity estimation , 2008, J. Electronic Imaging.

[66]  O. Bertrand,et al.  Oscillatory gamma activity in humans: a possible role for object representation. , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[67]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[68]  Joost van de Weijer,et al.  Color constancy based on the Grey-edge hypothesis , 2005, IEEE International Conference on Image Processing 2005.

[69]  Mark S. Drew,et al.  Improved machine learning for image category recognition by local color constancy , 2010, 2010 IEEE International Conference on Image Processing.

[70]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Arnold W. M. Smeulders,et al.  Color Based Object Recognition , 1997, ICIAP.

[72]  Brian V. Funt,et al.  Multispectral Colour Constancy , 2006, Color Imaging Conference.

[73]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[74]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[75]  Fabrizio Vagni,et al.  Survey of Hyperspectral and Multispectral Imaging Technologies (Etude sur les technologies d'imagerie hyperspectrale et multispectrale) , 2007 .