Constrained Deep Weak Supervision for Histopathology Image Segmentation

In this paper, we develop a new weakly supervised learning algorithm to learn to segment cancerous regions in histopathology images. This paper is under a multiple instance learning (MIL) framework with a new formulation, deep weak supervision (DWS); we also propose an effective way to introduce constraints to our neural networks to assist the learning process. The contributions of our algorithm are threefold: 1) we build an end-to-end learning system that segments cancerous regions with fully convolutional networks (FCNs) in which image-to-image weakly-supervised learning is performed; 2) we develop a DWS formulation to exploit multi-scale learning under weak supervision within FCNs; and 3) constraints about positive instances are introduced in our approach to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. The proposed algorithm, abbreviated as DWS-MIL, is easy to implement and can be trained efficiently. Our system demonstrates the state-of-the-art results on large-scale histopathology image data sets and can be applied to various applications in medical imaging beyond histopathology images, such as MRI, CT, and ultrasound images.

[1]  Mihail C. Roco,et al.  Converging Technologies for Improving Human Performance , 2003 .

[2]  Trevor Darrell,et al.  Fully Convolutional Multi-Class Multiple Instance Learning , 2014, ICLR.

[3]  Zhuowen Tu,et al.  Max-Margin Multiple-Instance Dictionary Learning , 2013, ICML.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Mikhail Teverovskiy,et al.  Multifeature Prostate Cancer Diagnosis and Gleason Grading of Histological Images , 2007, IEEE Transactions on Medical Imaging.

[6]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[8]  Zhuowen Tu,et al.  Multiple clustered instance learning for histopathology cancer image classification, segmentation and clustering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Zhuowen Tu,et al.  Auto-context and its application to high-level vision tasks , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Antonio Criminisi,et al.  Decision Forests for Computer Vision and Medical Image Analysis , 2013, Advances in Computer Vision and Pattern Recognition.

[13]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[14]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[15]  Hao Chen,et al.  Automated mitosis detection with deep regression networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[16]  George Papandreou,et al.  Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Bilwaj Gaonkar,et al.  A superpixel-based framework for automatic tumor segmentation on breast DCE-MRI , 2015, Medical Imaging.

[18]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Cordelia Schmid,et al.  Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yongyi Yang,et al.  Computer-Aided Detection and Diagnosis of Breast Cancer With Mammography: Recent Advances , 2009, IEEE Transactions on Information Technology in Biomedicine.

[21]  Luc Van Gool,et al.  Weakly Supervised Cascaded Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[23]  Joel H. Saltz,et al.  Efficient Multiple Instance Convolutional Neural Networks for Gigapixel Resolution Image Classification , 2015, ArXiv.

[24]  Zhuowen Tu,et al.  Contexts-Constrained Multiple Instance Learning for Histopathology Image Analysis , 2012 .

[25]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[26]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[27]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Zhuowen Tu,et al.  Unsupervised object class discovery via saliency-guided multiple class learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[30]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Oded Maron,et al.  Multiple-Instance Learning for Natural Scene Classification , 1998, ICML.

[32]  Kim L. Boyer,et al.  Computer-aided evaluation of neuroblastoma on whole-slide histology images: Classifying grade of neuroblastic differentiation , 2009, Pattern Recognit..

[33]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[34]  Yan Xu,et al.  Deep learning of feature representation with multiple instance learning for medical image analysis , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[36]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[37]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Hai Su,et al.  Deep Voting: A Robust Approach Toward Nucleus Localization in Microscopy Images , 2015, MICCAI.

[39]  Po-Whei Huang,et al.  Automatic Classification for Pathological Prostate Images Based on Fractal Analysis , 2009, IEEE Transactions on Medical Imaging.

[40]  Bayan S. Sharif,et al.  Fractal analysis in the detection of colonic cancer images , 2002, IEEE Transactions on Information Technology in Biomedicine.

[41]  Joel H. Saltz,et al.  Center-Focusing Multi-task CNN with Injected Features for Classification of Glioma Nuclear Images , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[42]  Fang Zhang,et al.  Deep convolutional activation features for large scale Brain Tumor histopathology image classification and segmentation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Ronan Collobert,et al.  From image-level to pixel-level labeling with Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Sun Young Park,et al.  Domain-Specific Image Analysis for Cervical Neoplasia Detection Based on Conditional Random Fields , 2011, IEEE Transactions on Medical Imaging.

[45]  Lin Yang,et al.  Transfer Shape Modeling Towards High-Throughput Microscopy Image Segmentation , 2016, MICCAI.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  A. Madabhushi.  Digital pathology image analysis: opportunities and challenges. , 2009, Imaging in medicine.

[48]  Shu Liao,et al.  Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition , 2016, IEEE Transactions on Medical Imaging.

[49]  Zhuowen Tu,et al.  Harvesting Mid-level Visual Concepts from Large-Scale Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Guang Yang,et al.  Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI , 2016, International Journal of Computer Assisted Radiology and Surgery.

[51]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.