Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images

Histology images are inherently symmetric under rotation, where each orientation is equally as likely to appear. However, this rotational symmetry is not widely utilised as prior knowledge in modern Convolutional Neural Networks (CNNs), resulting in data hungry models that learn independent features at each orientation. Allowing CNNs to be rotation-equivariant removes the necessity to learn this set of transformations from the data and instead frees up model capacity, allowing more discriminative features to be learned. This reduction in the number of required parameters also reduces the risk of overfitting. In this paper, we propose Dense Steerable Filter CNNs (DSF-CNNs) that use group convolutions with multiple rotated copies of each filter in a densely connected framework. Each filter is defined as a linear combination of steerable basis filters, enabling exact rotation and decreasing the number of trainable parameters compared to standard filters. We also provide the first in-depth comparison of different rotation-equivariant CNNs for histology image analysis and demonstrate the advantage of encoding rotational symmetry into modern architectures. We show that DSF-CNNs achieve state-of-the-art performance, with significantly fewer parameters, when applied to three different tasks in the area of computational pathology: breast tumour classification, colon gland segmentation and multi-tissue nuclear segmentation.

[1]  Max Welling,et al.  HexaConv , 2018, 1803.02108.

[2]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Qiang Qiu,et al.  Oriented Response Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  A. Robert Calderbank,et al.  RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks , 2018, ICLR.

[5]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[7]  Yee-Wah Tsang,et al.  Validation of digital pathology imaging for primary histopathological diagnosis , 2016, Histopathology.

[8]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[10]  T. Hermanns,et al.  Automated Gleason grading of prostate cancer tissue microarrays via deep learning , 2018, Scientific Reports.

[11]  Hao Chen,et al.  MILD‐Net: Minimal information loss dilated network for gland instance segmentation in colon histology images , 2018, Medical Image Anal..

[12]  Geert J. S. Litjens,et al.  Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology , 2019, Medical Image Anal..

[13]  Yair Weiss,et al.  Why do deep convolutional networks generalize so poorly to small image transformations? , 2018, J. Mach. Learn. Res..

[14]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Nasir M. Rajpoot,et al.  Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images , 2019, IEEE Transactions on Medical Imaging.

[17]  Guoying Zhao,et al.  SRN: Side-Output Residual Network for Object Symmetry Detection in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  David B. A. Epstein,et al.  Rota-Net: Rotation Equivariant Network for Simultaneous Gland and Lumen Segmentation in Colon Histology Images , 2019, ECDP.

[19]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[20]  Yoshua Bengio,et al.  An empirical evaluation of deep architectures on problems with many factors of variation , 2007, ICML '07.

[21]  Remco Duits,et al.  Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysis , 2020, Medical Image Anal..

[22]  Reka Hollandi,et al.  Test-time augmentation for deep learning-based cell segmentation on microscopy images , 2019, Scientific Reports.

[23]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Max Welling,et al.  Rotation Equivariant CNNs for Digital Pathology , 2018, MICCAI.

[25]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[26]  Nasir M. Rajpoot,et al.  Leveraging Unlabeled Whole-Slide-Images for Mitosis Detection , 2018, COMPAY/OMIA@MICCAI.

[27]  Maurice Weiler,et al.  Learning Steerable Filters for Rotation Equivariant CNNs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[29]  Surabhi Bhargava,et al.  A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology , 2017, IEEE Transactions on Medical Imaging.

[30]  Pheng-Ann Heng,et al.  CIA-Net: Robust Nuclei Instance Segmentation with Contour-aware Information Aggregation , 2019, IPMI.

[31]  Mitko Veta,et al.  Roto-Translation Covariant Convolutional Networks for Medical Image Analysis , 2018, MICCAI.

[32]  Nasir M. Rajpoot,et al.  Classification of lung cancer histology images using patch-level summary statistics , 2018, Medical Imaging.

[33]  Stephan J. Garbin,et al.  Harmonic Networks: Deep Translation and Rotation Equivariance , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Thomas Walter,et al.  Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map , 2019, IEEE Transactions on Medical Imaging.

[35]  Max Welling,et al.  Steerable CNNs , 2016, ICLR.

[36]  David B. A. Epstein,et al.  Micro‐Net: A unified model for segmentation of various objects in microscopy images , 2018, Medical Image Anal..

[37]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[40]  Joachim M. Buhmann,et al.  TI-POOLING: Transformation-Invariant Pooling for Feature Learning in Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Nikos Komodakis,et al.  Rotation Equivariant Vector Field Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[42]  Maurice Weiler,et al.  General E(2)-Equivariant Steerable CNNs , 2019, NeurIPS.

[43]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[44]  Max Welling,et al.  Sample Efficient Semantic Segmentation using Rotation Equivariant Convolutional Networks , 2018, ArXiv.

[45]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[46]  Jin Tae Kwak,et al.  Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images , 2018, Medical Image Anal..