Spectral reflectance estimation of organic tissue for improved color correction of video-assisted surgery

Abstract. With video-assisted surgery devices becoming more common in all fields of diagnostics and therapy, the question of how well such systems are able to reproduce surface colors of organic tissue arises, especially in cases where a proper distinction of different kinds of tissue—not only through their texture but also through their color—might be crucial for the success of a surgery. Since modern devices are usually made of a highly efficient, multispectral LED light source in combination with some light-guiding structures and a digital camera system, an approach of optimizing these systems’ color reproduction properties based on the estimation of in-situ spectral reflectances is proposed. Following the International Organization for Standardization (ISO) standard procedure of colorimetric characterization, an initial color correction matrix was determined first by solving a linear least-mean-squares optimization problem for a small set of artificial color samples mapping the corresponding camera responses onto the samples’ tristimulus values. This initial matrix was then used as a starting point for a second, nonlinear optimization, which makes use of the estimated reflectance spectra in order to minimize the average squared color difference between the color corrected and the actually perceived tristimulus values of the individual organic tissue samples. Compared to the ISO standard, which only knows simple color patches to be used for the nonlinear optimization, a significant enhancement in the color reproduction of organic tissue could be confirmed with the newly proposed method being applied instead.

[1]  Peter Morovic,et al.  Metamer-set-based approach to estimating surface reflectance from camera RGB. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[2]  Walter Gish,et al.  Camera Color Correction Using Two-Dimensional Transforms , 2013, Color Imaging Conference.

[3]  Gaurav Sharma Target-less Scanner Color Calibration , 1999, Color Imaging Conference.

[4]  Gaurav Sharma Set theoretic estimation for problems in subtractive color , 2000 .

[5]  S. Shapiro,et al.  An Approximate Analysis of Variance Test for Normality , 1972 .

[6]  Peter A. Rhodes,et al.  A study of digital camera colorimetric characterisation based on polynomial modelling , 2001 .

[7]  A. Elhan,et al.  Investigation of Four Different Normality Tests in Terms of Type 1 Error Rate and Power under Different Distributions , 2006 .

[8]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[9]  Jaume Pujol,et al.  Concerning the calculation of the color gamut in a digital camera , 2006 .

[10]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[11]  M. Hauta-Kasari,et al.  Wiener estimation method in estimating of spectral reflectance from RGB images , 2007, Pattern Recognition and Image Analysis.

[12]  Changjun Li,et al.  Characterization of trichromatic color cameras by using a new multispectral imaging technique. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[13]  Hui-Liang Shen,et al.  Improved reflectance reconstruction for multispectral imaging by combining different techniques. , 2007, Optics express.

[14]  Brian A Wandell,et al.  Spectral estimation theory: beyond linear but before Bayesian. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Shudao Zhou,et al.  Blurred image restoration using knife-edge function and optimal window Wiener filtering , 2018, PloS one.

[16]  Izumi Nishidate,et al.  Estimation of Melanin and Hemoglobin Using Spectral Reflectance Images Reconstructed from a Digital RGB Image by the Wiener Estimation Method , 2013, Sensors.

[17]  Morton B. Brown,et al.  Robust Tests for the Equality of Variances , 1974 .

[18]  Changjun Li,et al.  The Estimation of Spectral Reflectances Using the Smoothest Constraint Condition , 2001, Color Imaging Conference.

[19]  W. Mokrzycki,et al.  Color difference ΔE : a survey , 2011 .

[20]  Mark S. Drew,et al.  Constrained least-squares regression in color spaces , 1997, J. Electronic Imaging.

[21]  B. L. Welch THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL , 1938 .

[22]  Gaurav Sharma Digital Color Imaging Handbook , 2002 .

[23]  Glenn Healey,et al.  Using reflectance models for color scanner calibration. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Sabine Süsstrunk,et al.  What is the space of spectral sensitivity functions for digital color cameras? , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[25]  C. Tsui,et al.  Minimally invasive surgery: national trends in adoption and future directions for hospital strategy , 2013, Surgical Endoscopy.

[26]  Stephen Westland,et al.  A comparative study of the characterisation of colour cameras by means of neural networks and polynomial transforms , 2004 .

[27]  Michal Mackiewicz,et al.  Color Correction Using Root-Polynomial Regression , 2015, IEEE Transactions on Image Processing.

[28]  Hui-Liang Shen,et al.  Reflectance reconstruction for multispectral imaging by adaptive Wiener estimation. , 2007, Optics express.

[29]  Masahiro Yamaguchi,et al.  Maximum a posteriori estimation of spectral reflectance from color image and multipoint spectral measurements. , 2007, Applied optics.

[30]  Ronald A. Siegel,et al.  Fundamentals and applications of polymer gels: PPF-7 microsymposium, Oaxaca City, Mexico December 5-7, 2001. Preface , 2004 .

[31]  Norimichi Tsumura,et al.  Spectral color imaging system for estimating spectral reflectance of paint , 2007 .

[32]  Stephen R Grobmyer,et al.  Utilization of minimally invasive breast biopsy for the evaluation of suspicious breast lesions. , 2011, American journal of surgery.

[33]  Brian V. Funt,et al.  Camera characterization for color research , 2002 .

[34]  Roy S. Berns,et al.  A Spatially AdaptiveWiener Filter for Reflectance Estimation , 2008, Color Imaging Conference.

[35]  H Haneishi,et al.  System design for accurately estimating the spectral reflectance of art paintings. , 2000, Applied optics.

[36]  S. Naveen,et al.  Image Denoising by Fourier Block Processing and Wiener Filtering , 2015 .

[37]  Ara Darzi,et al.  Is minimally invasive surgery beneficial in the management of esophageal cancer? A meta-analysis , 2010, Surgical Endoscopy.

[38]  Noriyuki Shimano,et al.  Recovery of spectral reflectances of objects being imaged by multispectral cameras. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  T. Robinson,et al.  Minimally invasive surgery , 1999, European Surgical Research.

[40]  Mark S. Drew,et al.  Matrix Calculations for Digital Photography , 1997, Color Imaging Conference.

[41]  Welch Bl THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[42]  Michael S. Brown,et al.  Nonuniform Lattice Regression for Modeling the Camera Imaging Pipeline , 2012, ECCV.

[43]  Philipp Urban,et al.  Recovering Camera Sensitivities using Target-based Reflectances Captured under multiple LED-Illuminations , 2010 .

[44]  L. Maloney,et al.  Color constancy: a method for recovering surface spectral reflectance , 1987 .

[45]  Mo Zhou,et al.  Hospital cost implications of increased use of minimally invasive surgery. , 2015, JAMA surgery.

[46]  Philipp Urban,et al.  Spectral image reconstruction using an edge preserving spatio-spectral Wiener estimation. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[47]  Tony Johnson,et al.  Methods for characterizing colour scanners and digital cameras , 1996 .

[48]  Quan Liu,et al.  Modified Wiener estimation of diffuse reflectance spectra from RGB values by the synthesis of new colors for tissue measurements. , 2012, Journal of biomedical optics.

[49]  Xiandou Zhang,et al.  Reconstructing spectral reflectance by dividing spectral space and extending the principal components in principal component analysis. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  Norimichi Tsumura,et al.  Restoration of Noisy Images using Wiener Filters Designed in Color Space , 2000, PICS.