Joint desmoking, specularity removal, and denoising of laparoscopy images via graphical models and Bayesian inference

Laparoscopic images exhibit artifacts resulting from surgical smoke, specular highlights, and noise. These artifacts degrade the results of subsequent processing (e.g., tracking, segmentation, and depth analysis) and compromise surgical quality. We formulate a unified Bayesian inference problem for desmoking, specularity removal, and denoising in laparoscopic images. We propose novel probabilistic graphical models and sparse dictionary models as image priors. For inference, we rely on variational Bayesian expectation maximization. Results on simulated and real-world laparoscopic images, including clinical expert evaluation, show that our joint optimization method outperforms the state of the art.

[1]  Michael F. Cohen,et al.  Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[2]  Suyash P. Awate,et al.  Joint desmoking and denoising of laparoscopy images , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[3]  Peyman Milanfar,et al.  Removal of haze and noise from a single image , 2012, Electronic Imaging.

[4]  Wonha,et al.  Single image dehazing using segmenting dark channel prior , 2014 .

[5]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Zheng Guo,et al.  Improved single image dehazing using guided filter , 2011 .

[7]  W L Barrett,et al.  Surgical smoke: a review of the literature. Is this just a lot of hot air? , 2003, Surgical endoscopy.

[8]  Suyash P. Awate,et al.  Higher-order image statistics for unsupervised, information-theoretic, adaptive, image filtering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Adrien Bartoli,et al.  Automatic Detection of the Uterus and Fallopian Tube Junctions in Laparoscopic Images , 2015, IPMI.

[10]  Farida Cheriet,et al.  Detection and correction of specular reflections for automatic surgical tool segmentation in thoracoscopic images , 2007, Machine Vision and Applications.

[11]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[12]  Philippe Cinquin,et al.  3D Tracking of Laparoscopic Instruments Using Statistical and Geometric Modeling , 2011, MICCAI.

[13]  Lena Maier-Hein,et al.  Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery , 2013, Medical Image Anal..

[14]  Gerard Lacey,et al.  Automatic Segmentation and Inpainting of Specular Highlights for Endoscopic Imaging , 2010, EURASIP J. Image Video Process..

[15]  Suyash P. Awate,et al.  Feature-Preserving MRI Denoising: A Nonparametric Empirical Bayes Approach , 2007, IEEE Transactions on Medical Imaging.

[16]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Ko Nishino,et al.  Bayesian Defogging , 2012, International Journal of Computer Vision.

[18]  Guang-Zhong Yang,et al.  Probabilistic Tracking of Affine-Invariant Anisotropic Regions , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Guang-Zhong Yang,et al.  Removing specular reflection components for robotic assisted laparoscopic surgery , 2005, IEEE International Conference on Image Processing 2005.

[20]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Lena Maier-Hein,et al.  Comparative Validation of Single-Shot Optical Techniques for Laparoscopic 3-D Surface Reconstruction , 2014, IEEE Transactions on Medical Imaging.

[22]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[23]  Jean-Alexandre Long,et al.  Automatic detection of instruments in laparoscopic images: a first step towards high level command of robotized endoscopic holders , 2006 .

[24]  Luc Soler,et al.  An Augmented Reality Framework for Soft Tissue Surgery , 2014, MICCAI.

[25]  Truong Q. Nguyen,et al.  Fast single image fog removal using the adaptive Wiener filter , 2013, 2013 IEEE International Conference on Image Processing.

[26]  Danail Stoyanov,et al.  Surgical Vision , 2011, Annals of Biomedical Engineering.