A Solution of the Dichromatic Model for Multispectral Photometric Invariance

In this paper, we address the problem of photometric invariance in multispectral imaging making use of an optimisation approach based upon the dichromatic model. In this manner, we cast the problem of recovering the spectra of the illuminant, the surface reflectance and the shading and specular factors in a structural optimisation setting. Making use of the additional information provided by multispectral imaging and the structure of image patches, we recover the dichromatic parameters of the scene. To do this, we formulate a target cost function combining the dichromatic error and the smoothness priors for the surfaces under study. The dichromatic parameters are recovered through minimising this cost function in a coordinate descent manner. The algorithm is quite general in nature, admitting the enforcement of smoothness constraints and extending in a straightforward manner to trichromatic settings. Moreover, the objective function is convex with respect to the subset of variables to be optimised in each alternating step of the minimisation strategy. This gives rise to an optimal closed-form solution for each of the iterations in our algorithm. We illustrate the effectiveness of our method for purposes of illuminant spectrum recovery, skin recognition, material clustering and specularity removal. We also compare our results to a number of alternatives.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  B. Wandell,et al.  Standard surface-reflectance model and illuminant estimation , 1989 .

[3]  Stan Z. Li,et al.  Discontinuous MRF prior and robust statistics: a comparative study , 1995, Image Vis. Comput..

[4]  T. Poggio,et al.  Ill-Posed Problems and Regularization Analysis in Early Vision , 1984 .

[5]  Edwin R. Hancock,et al.  A probabilistic framework for specular shape-from-shading , 2003, Pattern Recognit..

[6]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[7]  Shree K. Nayar,et al.  Computing reflectance ratios from an image , 1993, Pattern Recognit..

[8]  Shree K. Nayar,et al.  A class of photometric invariants: separating material from shape and illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Brian V. Funt,et al.  A data set for color research , 2002 .

[10]  E. Adelson,et al.  Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination , 2001 .

[11]  Glenn Healey,et al.  Object recognition using invariant profiles , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Huang Yumin,et al.  A PHYSICAL APPROACH TO COLOR IMAGE UNDERSTANDING , 1991 .

[13]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[14]  Frank P. Ferrie,et al.  Curvature consistency improves local shading analysis , 1992, CVGIP Image Underst..

[15]  Edwin R. Hancock,et al.  New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Glenn Healey,et al.  Invariant recognition in hyperspectral images , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[18]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2006, International Journal of Computer Vision.

[19]  Edwin R. Hancock,et al.  Highlight Removal Using Shape-from-Shading , 2002, ECCV.

[20]  Katsushi Ikeuchi,et al.  Separating reflection components based on chromaticity and noise analysis , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  B. Wandell,et al.  Component estimation of surface spectral reflectance , 1990 .

[22]  H C Lee,et al.  Method for computing the scene-illuminant chromaticity from specular highlights. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[23]  E. Land Recent advances in retinex theory , 1986, Vision Research.

[24]  Ramesh Raskar,et al.  Introduction , 2006, SIGGRAPH Courses.

[25]  Thomas S. Huang,et al.  Image processing , 1971 .

[26]  Glenn Healey,et al.  Invariant mixture recognition in hyperspectral images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  Elli Angelopoulou,et al.  Objective Colour from Multispectral Imaging , 2000, ECCV.

[28]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[29]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[30]  David J. Kriegman,et al.  Specularity Removal in Images and Videos: A PDE Approach , 2006, ECCV.

[31]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[32]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  Andrew Blake,et al.  Detecting Specular Reflections Using Lambertian Constraints , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[35]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[36]  Edward H. Adelson,et al.  Estimating surface reflectance properties from images under unknown illumination , 2001, IS&T/SPIE Electronic Imaging.

[37]  Gerald Schaefer,et al.  Solving for Colour Constancy using a Constrained Dichromatic Reflection Model , 2001, International Journal of Computer Vision.

[38]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Glenn Healey,et al.  Estimating spectral reflectance using highlights , 1991, Image Vis. Comput..

[41]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[42]  Peter B. Delahunt,et al.  Bayesian model of human color constancy. , 2006, Journal of vision.

[43]  Elli Angelopoulou,et al.  Specular Highlight Detection Based on the Fresnel Reflection Coefficient , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[44]  Abdesselam Bouzerdoum,et al.  Skin segmentation using color pixel classification: analysis and comparison , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Michael J. Brooks,et al.  Shape and Source from Shading , 1985, IJCAI.

[46]  Stephen Lin,et al.  Separation of diffuse and specular reflection in color images , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[47]  Gerald Schaefer,et al.  Convex and non-convex illuminant constraints for dichromatic colour constancy , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[48]  Joost van de Weijer,et al.  Color constancy based on the Grey-edge hypothesis , 2005, IEEE International Conference on Image Processing 2005.

[49]  D H Brainard,et al.  Analysis of the retinex theory of color vision. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[50]  Frederick Mosteller,et al.  Understanding Robust and Exploratory Data Analysis. , 1983 .

[51]  Theo Gevers,et al.  Detection and Classification of Hyper-Spectral Edges , 1999, BMVC.

[52]  Brian V. Funt,et al.  Colour by Correlation in a Three-Dimensional Colour Space , 2000, ECCV.

[53]  Ramesh Raskar,et al.  Computational photography , 2005, Eurographics.

[54]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[55]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[56]  F. Mosteller,et al.  Understanding robust and exploratory data analysis , 1985 .

[57]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[58]  Gerald Schaefer,et al.  A combined physical and statistical approach to colour constancy , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[59]  Takeo Kanade,et al.  The measurement of highlights in color images , 1988, International Journal of Computer Vision.

[60]  Graham D. Finlayson,et al.  Improving gamut mapping color constancy , 2000, IEEE Trans. Image Process..

[61]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[62]  Carle M. Pieters,et al.  Deconvolution of mineral absorption bands: An improved approach , 1990 .

[63]  David J. Kriegman,et al.  Color Subspaces as Photometric Invariants , 2006, CVPR.

[64]  Kristin J. Dana,et al.  Relief texture from specularities , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[67]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[68]  Frederick Mosteller,et al.  Understanding robust and exploratory data analysis , 1983 .

[69]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.