Interactive segmentation of medical images through fully convolutional neural networks

Image segmentation plays an essential role in medicine for both diagnostic and interventional tasks. Segmentation approaches are either manual, semi-automated or fully-automated. Manual segmentation offers full control over the quality of the results, but is tedious, time consuming and prone to operator bias. Fully automated methods require no human effort, but often deliver sub-optimal results without providing users with the means to make corrections. Semi-automated approaches keep users in control of the results by providing means for interaction, but the main challenge is to offer a good trade-off between precision and required interaction. In this paper we present a deep learning (DL) based semi-automated segmentation approach that aims to be a "smart" interactive tool for region of interest delineation in medical images. We demonstrate its use for segmenting multiple organs on computed tomography (CT) of the abdomen. Our approach solves some of the most pressing clinical challenges: (i) it requires only one to a few user clicks to deliver excellent 2D segmentations in a fast and reliable fashion; (ii) it can generalize to previously unseen structures and "corner cases"; (iii) it delivers results that can be corrected quickly in a smart and intuitive way up to an arbitrary degree of precision chosen by the user and (iv) ensures high accuracy. We present our approach and compare it to other techniques and previous work to show the advantages brought by our method.

[1]  Supun Samarasekera,et al.  Fuzzy Connectedness and Object Definition: Theory, Algorithms, and Applications in Image Segmentation , 1996, CVGIP Graph. Model. Image Process..

[2]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Arnold W. M. Smeulders,et al.  Interaction in the segmentation of medical images: A survey , 2001, Medical Image Anal..

[4]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[5]  Rüdiger Westermann,et al.  Random Walks for Interactive Organ Segmentation in Two and Three Dimensions: Implementation and Validation , 2005, MICCAI.

[6]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[7]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[8]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[9]  M. Giger,et al.  Anniversary paper: History and status of CAD and quantitative image analysis: the role of Medical Physics and AAPM. , 2008, Medical physics.

[10]  Caroline Petitjean,et al.  A review of segmentation methods in short axis cardiac MR images , 2011, Medical Image Anal..

[11]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[12]  David Zhang,et al.  A survey of graph theoretical approaches to image segmentation , 2013, Pattern Recognit..

[13]  Ulas Bagci,et al.  A review on segmentation of positron emission tomography images , 2014, Comput. Biol. Medicine.

[14]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[17]  Joshua D. Warner,et al.  Semi-automated segmentation of pre-operative low grade gliomas in magnetic resonance imaging , 2015, Cancer Imaging.

[18]  Fredrik Kahl,et al.  Deepseg: Abdominal Organ Segmentation Using Deep Convolutional Neural Networks , 2016 .

[19]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[20]  Marie E. Edwards,et al.  Semiautomated Segmentation of Polycystic Kidneys in T2-Weighted MR Images. , 2016, AJR. American journal of roentgenology.

[21]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[22]  Sébastien Ourselin,et al.  Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views , 2016, Medical Image Anal..

[23]  Ning Xu,et al.  Deep Interactive Object Selection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Qiong Yang,et al.  Accuracy and efficiency of computer-aided anatomical analysis using 3D visualization software based on semi-automated and automated segmentations. , 2017, Annals of anatomy = Anatomischer Anzeiger : official organ of the Anatomische Gesellschaft.

[25]  Sanja Fidler,et al.  Annotating Object Instances with a Polygon-RNN , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Bradley J. Erickson,et al.  Performance of an Artificial Multi-observer Deep Neural Network for Fully Automated Segmentation of Polycystic Kidneys , 2017, Journal of Digital Imaging.

[27]  B. van Ginneken,et al.  Use of Volumetry for Lung Nodule Management: Theory and Practice. , 2017, Radiology.

[28]  Ben Glocker,et al.  DLTK: State of the Art Reference Implementations for Deep Learning on Medical Images , 2017, ArXiv.

[29]  Ender Konukoglu,et al.  Learning to Segment Medical Images with Scribble-Supervision Alone , 2018, DLMIA/ML-CDS@MICCAI.

[30]  Sébastien Ourselin,et al.  Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning , 2017, IEEE Transactions on Medical Imaging.

[31]  Klaus H. Maier-Hein,et al.  nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2018, Bildverarbeitung für die Medizin.

[32]  Yuichiro Hayashi,et al.  A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation , 2018, MICCAI.

[33]  Luc Van Gool,et al.  Deep Extreme Cut: From Extreme Points to Object Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Eirikur Agustsson,et al.  Interactive Full Image Segmentation , 2018, ArXiv.

[35]  Ronald M. Summers,et al.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms , 2019, ArXiv.

[36]  Sébastien Ourselin,et al.  DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Einly Lim,et al.  Segmentation of left ventricle in late gadolinium enhanced MRI through 2D‐4D registration for infarct localization in 3D patient‐specific left ventricular model , 2018, Magnetic resonance in medicine.