ARA: accurate, reliable and active histopathological image classification framework with Bayesian deep learning

Machine learning algorithms hold the promise to effectively automate the analysis of histopathological images that are routinely generated in clinical practice. Any machine learning method used in the clinical diagnostic process has to be extremely accurate and, ideally, provide a measure of uncertainty for its predictions. Such accurate and reliable classifiers need enough labelled data for training, which requires time-consuming and costly manual annotation by pathologists. Thus, it is critical to minimise the amount of data needed to reach the desired accuracy by maximising the efficiency of training. We propose an accurate, reliable and active (ARA) image classification framework and introduce a new Bayesian Convolutional Neural Network (ARA-CNN) for classifying histopathological images of colorectal cancer. The model achieves exceptional classification accuracy, outperforming other models trained on the same dataset. The network outputs an uncertainty measurement for each tested image. We show that uncertainty measures can be used to detect mislabelled training samples and can be employed in an efficient active learning workflow. Using a variational dropout-based entropy measure of uncertainty in the workflow speeds up the learning process by roughly 45%. Finally, we utilise our model to segment whole-slide images of colorectal tissue and compute segmentation-based spatial statistics.

[1]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.

[3]  Masahiro Murakawa,et al.  Gastric Pathology Image Classification Using Stepwise Fine-Tuning for Deep Neural Networks , 2018, Journal of healthcare engineering.

[4]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[5]  Tuan D. Pham,et al.  Scaling of Texture in Training Autoencoders for Classification of Histological Images of Colorectal Cancer , 2017, ISNN.

[6]  Francesco Bianconi,et al.  Multi-class texture analysis in colorectal cancer histology , 2016, Scientific Reports.

[7]  George Lee,et al.  Image analysis and machine learning in digital pathology: Challenges and opportunities , 2016, Medical Image Anal..

[8]  Fabio A. González,et al.  A Deep Learning Architecture for Image Representation, Visual Interpretability and Automated Basal-Cell Carcinoma Cancer Detection , 2013, MICCAI.

[9]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[10]  Bing Lang,et al.  Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers , 2017, Deep Learning and Convolutional Neural Networks for Medical Image Computing.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[13]  Scott T. Acton,et al.  SDL: Saliency-Based Dictionary Learning Framework for Image Similarity , 2018, IEEE Transactions on Image Processing.

[14]  Yarin Gal,et al.  Understanding Measures of Uncertainty for Adversarial Example Detection , 2018, UAI.

[15]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[16]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[17]  Yinyin Yuan Spatial Heterogeneity in the Tumor Microenvironment. , 2016, Cold Spring Harbor perspectives in medicine.

[18]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[21]  Zoubin Ghahramani,et al.  Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference , 2015, ArXiv.

[22]  Luiz Eduardo Soares de Oliveira,et al.  A Dataset for Breast Cancer Histopathological Image Classification , 2016, IEEE Transactions on Biomedical Engineering.

[23]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[24]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[25]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Juho Kannala,et al.  Deep learning for magnification independent breast cancer histopathology image classification , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[27]  Dinggang Shen,et al.  Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.

[28]  Siegfried Wahl,et al.  Leveraging uncertainty information from deep neural networks for disease detection , 2016, Scientific Reports.

[29]  Shihui Ying,et al.  Histopathological image classification with bilinear convolutional neural networks , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  Olaf Hellwich,et al.  Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology , 2017, Comput. Medical Imaging Graph..

[31]  Pedro Costa,et al.  MedAL: Accurate and Robust Deep Active Learning for Medical Image Analysis , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[32]  Alessandro Santana Martins,et al.  Classification of colorectal cancer based on the association of multidimensional and multiresolution features , 2019, Expert Syst. Appl..

[33]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[34]  Yuanjie Zheng,et al.  Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model , 2017, Scientific Reports.

[35]  A. Kiureghian,et al.  Aleatory or epistemic? Does it matter? , 2009 .

[36]  Wei Liu,et al.  Scalable Histopathological Image Analysis via Active Learning , 2014, MICCAI.

[37]  Zhipeng Jia,et al.  Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features , 2017, BMC Bioinformatics.

[38]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[39]  Sanghoon Lee,et al.  Interactive phenotyping of large-scale histology imaging data with HistomicsML , 2017, bioRxiv.

[40]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[41]  Bram van Ginneken,et al.  The importance of stain normalization in colorectal tissue classification with convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[42]  ImageNet Classification with Deep Convolutional Neural , 2013 .

[43]  Hai Su,et al.  Beyond Classification: Structured Regression for Robust Cell Detection Using Convolutional Neural Network , 2015, MICCAI.

[44]  Daoqiang Zhang,et al.  Deep active learning for nucleus classification in pathology images , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[45]  Baolin Du,et al.  Breast Cancer Histopathological Image Classification via Deep Active Learning and Confidence Boosting , 2018, ICANN.

[46]  H. Fox Is H&E morphology coming to an end? , 2000, Journal of clinical pathology.

[47]  A. Gazdar,et al.  Comprehensive analysis of lung cancer pathology images to discover tumor shape and boundary features that predict survival outcome , 2018, Scientific Reports.

[48]  Anant Madabhushi,et al.  A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images , 2016, Neurocomputing.

[49]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[50]  Lawrence Carin,et al.  An Active Learning Approach for Rapid Characterization of Endothelial Cells in Human Tumors , 2014, PloS one.

[51]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[52]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[53]  K. Aldape,et al.  Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care , 2017, npj Precision Oncology.

[54]  Anant Madabhushi,et al.  An active learning based classification strategy for the minority class problem: application to histopathology annotation , 2011, BMC Bioinformatics.

[55]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.