Weakly Supervised Organ Localization with Attention Maps Regularized by Local Area Reconstruction

Fully supervised methods with numerous dense-labeled training data have achieved accurate localization results for anatomical structures. However, obtaining such a dedicated dataset usually requires clinical expertise and time-consuming annotation process. In this work, we tackle the organ localization problem under the setting of image-level annotations. Previous Class Activation Map (CAM) and its derivatives have proved that discriminative regions of images can be located with basic classification networks. To improve the representative capacity of attention maps generated by CAMs, a novel learning-based Local Area Reconstruction (LAR) method is proposed. Our weakly supervised organ localization network, namely OLNet, can generate high-resolution attention maps that preserve fine-detailed target anatomical structures. Online generated pseudo ground-truth is utilized to impose geometric constraints on attention maps. Extensive experiments on In-house Chest CT Dataset and Kidney Tumor Segmentation Benchmark (KiTS19) show that our approach can provide promising localization results both in saliency map and semantic segmentation perspectives.

[1]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Nikolaos Papanikolopoulos,et al.  The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes , 2019, ArXiv.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Dorin Comaniciu,et al.  Fast Automatic Heart Chamber Segmentation from 3D CT Data Using Marginal Space Learning and Steerable Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Yongxiang Huang,et al.  CELNet: Evidence Localization for Pathology Images using Weakly Supervised Learning , 2019, MICCAI.

[8]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[9]  Konstantinos N. Plataniotis,et al.  HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Elsa D. Angelini,et al.  Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules , 2017, MICCAI.

[11]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Song Wang,et al.  Automatic localization of solid organs on 3D CT images by a collaborative majority voting decision based on ensemble learning , 2012, Comput. Medical Imaging Graph..

[13]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Gernot A. Fink,et al.  Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[15]  Antonio Criminisi,et al.  Regression forests for efficient anatomy detection and localization in computed tomography scans , 2013, Medical Image Anal..

[16]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[17]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[18]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[19]  Christoph H. Lampert,et al.  Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation , 2016, ECCV.

[20]  Konstantinos N. Plataniotis,et al.  A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains , 2019, International Journal of Computer Vision.