Detection and correction of specular reflections for automatic surgical tool segmentation in thoracoscopic images

This paper presents an algorithm that automatically detects and corrects specular reflections in thoracoscopic images and its application in the context of automatic segmentation of surgical tools. The detection is done by isolating the spike component of the specular reflection which is characterized by a bump at the end of the histogram of thoracoscopic images. The specular lobe is then extracted in the neighborhood of the spike component of the reflection. The result is a mask of the reflections positions in the image. Thereafter, the image is corrected using Oliveira et al.’s digital inpainting method. The automatic segmentation of surgical tools using the corrected images is then demonstrated. Results of the segmentation with and without the specular reflection elimination technique are compared. Moreover, 108 images extracted from 5 different surgeries performed under various conditions were considered to demonstrate the effectiveness of the proposed technique.

[1]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[2]  Shuchang Xu,et al.  Highlight Detection and Removal Based on Chromaticity , 2005, ICIAR.

[3]  Farida Cheriet,et al.  Bayesian Differentiation of Multi-scale Line-Structures for Model-Free Instrument Segmentation in Thoracoscopic Images , 2005, ICIAR.

[4]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[5]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[6]  Takayuki Okatani,et al.  Reconstructing shape from shading with a point light source at the projection center: shape reconstruction from an endoscope image , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[7]  Simon Masnou,et al.  Disocclusion: a variational approach using level lines , 2002, IEEE Trans. Image Process..

[8]  Lawrence B. Wolff,et al.  On the relative brightness of specular and diffuse reflection , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Heinrich Niemann,et al.  Making the Invisible Visible: Highlight Substitution by Color Light Fields , 2002, CGIV.

[11]  Farida Cheriet,et al.  Segmentation of Laparoscopic Images for Computer Assisted Surgery , 2003, SCIA.

[12]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[13]  Daniel H. Kim,et al.  Minimally invasive spinal surgery: a historical perspective. , 2004, Neurosurgical focus.

[14]  Fernando Torres Medina,et al.  Automatic Detection of Specular Reflectance in Colour Images Using the MS Diagram , 2003, CAIP.

[15]  Yoichi Miyake,et al.  Highlight Removal in Endoscope Images , 2006, CGIV.

[16]  Kai-Kuang Ma,et al.  Color distance histogram: a novel descriptor for color image segmentation , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[17]  Jae Byung Park.  Detection of Specular Highlights in Color Images using a New Color Space Transformation , 2004, 2004 IEEE International Conference on Robotics and Biomimetics.

[18]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[19]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[20]  S. Lin,et al.  Generation of diffuse and specular appearance from photometric images , 1999, Proceedings Workshop on Photometric Modeling for Computer Vision and Graphics (Cat. No.PR00271).

[21]  S. Mallat.  A wavelet tour of signal processing , 1998 .