Weakly Supervised Object Detection with 2D and 3D Regression Neural Networks

Finding automatically multiple lesions in large images is a common problem in medical image analysis. Solving this problem can be challenging if, during optimization, the automated method cannot access information about the location of the lesions nor is given single examples of the lesions. We propose a new weakly supervised detection method using neural networks, that computes attention maps revealing the locations of brain lesions. These attention maps are computed using the last feature maps of a segmentation network optimized only with global image-level labels. The proposed method can generate attention maps at full input resolution without need for interpolation during preprocessing, which allows small lesions to appear in attention maps. For comparison, we modify state-of-the-art methods to compute attention maps for weakly supervised object detection, by using a global regression objective instead of the more conventional classification objective. This regression objective optimizes the number of occurrences of the target object in an image, e.g. the number of brain lesions in a scan, or the number of digits in an image. We study the behavior of the proposed method in MNIST-based detection datasets, and evaluate it for the challenging detection of enlarged perivascular spaces - a type of brain lesion - in a dataset of 2202 3D scans with point-wise annotations in the center of all lesions in four brain regions. In MNIST-based datasets, the proposed method outperforms the other methods. In the brain dataset, the weakly supervised detection methods come close to the human intrarater agreement in each region. The proposed method reaches the best area under the curve in two out of four regions, and has the lowest number of false positive detections in all regions, while its average sensitivity over all regions is similar to that of the other best methods. The proposed method can facilitate epidemiological and clinical studies of enlarged perivascular spaces and help advance research in the etiology of enlarged perivascular spaces and in their relationship with cerebrovascular diseases.

[1]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[2]  Matthew N. Dailey,et al.  Multiple human tracking in high-density crowds , 2009, Image Vis. Comput..

[3]  Liang Chen,et al.  Attention-Gated Networks for Improving Ultrasound Scan Plane Detection , 2018, MICCAI 2018.

[4]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[5]  Quanshi Zhang,et al.  Interpretable Convolutional Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[9]  Nilanjan Ray,et al.  Cell Counting by Regression Using Convolutional Neural Network , 2016, ECCV Workshops.

[10]  Meritxell Bach Cuadra,et al.  A novel segmentation framework for uveal melanoma in magnetic resonance imaging based on class activation maps , 2019, MIDL.

[11]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[12]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[14]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[15]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Arthur W. Toga,et al.  Image processing approaches to enhance perivascular space visibility and quantification using MRI , 2019 .

[17]  Lucia Ballerini,et al.  Perivascular Spaces Segmentation in Brain MRI Using Optimal 3D Filtering , 2017, Scientific Reports.

[18]  Giovanni Montana,et al.  Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker , 2016, NeuroImage.

[19]  Nassir Navab,et al.  Deeper Depth Prediction with Fully Convolutional Residual Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[20]  Noam Alperin,et al.  Brain arterial dilatation modifies the association between extracranial pulsatile hemodynamics and brain perivascular spaces: the Northern Manhattan Study , 2019, Hypertension Research.

[21]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[22]  H. Rolf Jäger,et al.  3D multirater RCNN for multimodal multiclass detection and characterisation of extremely small objects , 2018, MIDL.

[23]  Mohammad Tariqul Islam,et al.  Machine learning approach of automatic identification and counting of blood cells , 2019, Healthcare technology letters.

[24]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[25]  Hai Su,et al.  Efficient and robust cell detection: A structured regression approach , 2018, Medical Image Anal..

[26]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[27]  S. Black,et al.  Understanding the role of the perivascular space in cerebral small vessel disease , 2018, Cardiovascular research.

[28]  Yoshua Bengio,et al.  Count-ception: Counting by Fully Convolutional Redundant Counting , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[29]  Been Kim,et al.  Sanity Checks for Saliency Maps , 2018, NeurIPS.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  E Kanal,et al.  Normal perivascular spaces mimicking lacunar infarction: MR imaging. , 1988, Radiology.

[32]  Nick C Fox,et al.  Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration , 2013, The Lancet Neurology.

[33]  Richard S. Zemel,et al.  End-to-End Instance Segmentation with Recurrent Attention , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Wesam A. Sakla,et al.  A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learning , 2016, ECCV.

[36]  David Gur,et al.  Area under the Free‐Response ROC Curve (FROC) and a Related Summary Index , 2009, Biometrics.

[37]  Hyo-Eun Kim,et al.  Self-Transfer Learning for Weakly Supervised Lesion Localization , 2016, MICCAI.

[38]  Grantham Pang,et al.  People Counting and Human Detection in a Challenging Situation , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[39]  Daniel L Schwartz,et al.  MR Imaging-based Multimodal Autoidentification of Perivascular Spaces (mMAPS): Automated Morphologic Segmentation of Enlarged Perivascular Spaces at Clinical Field Strength. , 2017, Radiology.

[40]  Meng Law,et al.  Image processing approaches to enhance perivascular space visibility and quantification using MRI , 2019, Scientific Reports.

[41]  Marleen de Bruijne,et al.  Enlarged perivascular spaces in brain MRI: Automated quantification in four regions , 2019, NeuroImage.

[42]  Shiming Xiang,et al.  Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks , 2014, IEEE Geoscience and Remote Sensing Letters.

[43]  Nicu Sebe,et al.  Self Paced Deep Learning for Weakly Supervised Object Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  J. Alison Noble,et al.  Weakly Supervised Learning of Placental Ultrasound Images with Residual Networks , 2017, MIUA.

[45]  Gustavo Carneiro,et al.  Model Agnostic Saliency For Weakly Supervised Lesion Detection From Breast DCE-MRI , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[46]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Wenyu Liu,et al.  PCL: Proposal Cluster Learning for Weakly Supervised Object Detection , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[49]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[50]  Kate Saenko,et al.  RISE: Randomized Input Sampling for Explanation of Black-box Models , 2018, BMVC.

[51]  A. Hofman,et al.  The Rotterdam Scan Study: design update 2016 and main findings , 2015, European Journal of Epidemiology.

[52]  Marleen de Bruijne,et al.  Deep Learning from Label Proportions for Emphysema Quantification , 2018, MICCAI.

[53]  Jun Zhang,et al.  Cells Counting with Convolutional Neural Network , 2018, ICIC.

[54]  Saeed Hassanpour,et al.  Looking Under the Hood: Deep Neural Network Visualization to Interpret Whole-Slide Image Analysis Outcomes for Colorectal Polyps , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[55]  Bernard Mazoyer,et al.  High dilated perivascular space burden: a new MRI marker for risk of intracerebral hemorrhage , 2019, Neurobiology of Aging.

[56]  Andrew Zisserman,et al.  Learning To Count Objects in Images , 2010, NIPS.

[57]  Aad van der Lugt,et al.  DETERMINANTS OF ENLARGED VIRCHOW-ROBIN SPACES: THE UNIVRSE CONSORTIUM , 2014, Alzheimer's & Dementia.

[58]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Loïc Le Folgoc,et al.  Attention U-Net: Learning Where to Look for the Pancreas , 2018, ArXiv.

[60]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[61]  Lior Wolf,et al.  Learning to Count with CNN Boosting , 2016, ECCV.

[62]  Jordi Vitrià,et al.  Learning to count with deep object features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[63]  Gustavo Carneiro,et al.  Lesion Saliency for Weakly Supervised Lesion Detection from Breast DCE-MRI , 2018 .

[64]  Benjamin F. J. Verhaaren,et al.  Rating Method for Dilated Virchow–Robin Spaces on Magnetic Resonance Imaging , 2013, Stroke.

[65]  Andrew Zisserman,et al.  Microscopy cell counting and detection with fully convolutional regression networks , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[66]  Marleen de Bruijne,et al.  Grey Matter Age Prediction as a Biomarker for Risk of Dementia: A Population-based Study , 2019, bioRxiv.

[67]  Marleen de Bruijne,et al.  3D regression neural network for the quantification of enlarged perivascular spaces in brain MRI , 2018, Medical Image Anal..

[68]  M. Sasikumar,et al.  Vehicle Detection and Classification from High Resolution Satellite Images , 2014 .

[69]  Yuan Xie,et al.  Weakly Supervised Salient Object Detection Using Image Labels , 2018, AAAI.

[70]  Georg Langs,et al.  f‐AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks , 2019, Medical Image Anal..

[71]  P. Elliott,et al.  UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age , 2015, PLoS medicine.

[72]  Marleen de Bruijne,et al.  GP-Unet: Lesion Detection from Weak Labels with a 3D Regression Network , 2017, MICCAI.

[73]  C. Sudlow,et al.  Enlarged perivascular spaces and cerebral small vessel disease , 2013, International journal of stroke : official journal of the International Stroke Society.

[74]  Claudia L. Satizabal,et al.  Effects of Arterial Stiffness on Brain Integrity in Young Adults From the Framingham Heart Study , 2016, Stroke.

[75]  Andrea Vedaldi,et al.  Weakly Supervised Deep Detection Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).