Thyroid Cancer Malignancy Prediction From Whole Slide Cytopathology Images

We consider preoperative prediction of thyroid cancer based on ultra-high-resolution whole-slide cytopathology images. Inspired by how human experts perform diagnosis, our approach first identifies and classifies diagnostic image regions containing informative thyroid cells, which only comprise a tiny fraction of the entire image. These local estimates are then aggregated into a single prediction of thyroid malignancy. Several unique characteristics of thyroid cytopathology guide our deep-learning-based approach. While our method is closely related to multiple-instance learning, it deviates from these methods by using a supervised procedure to extract diagnostically relevant regions. Moreover, we propose to simultaneously predict thyroid malignancy, as well as a diagnostic score assigned by a human expert, which further allows us to devise an improved training strategy. Experimental results show that the proposed algorithm achieves performance comparable to human experts, and demonstrate the potential of using the algorithm for screening and as an assistive tool for the improved diagnosis of indeterminate cases.

[1]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[2]  Fa Wu,et al.  Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks , 2017, International Journal of Computer Assisted Radiology and Surgery.

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Gustavo K. Rohde,et al.  Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning , 2014, Medical Image Anal..

[5]  Jinlian Ma,et al.  A pre‐trained convolutional neural network based method for thyroid nodule diagnosis , 2017, Ultrasonics.

[6]  Petros Karakitsos,et al.  Cascaded learning vector quantizer neural networks for the discrimination of thyroid lesions. , 2011, Analytical and quantitative cytology and histology.

[7]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[8]  Ya-Chen Tina Shih,et al.  The Clinical and Economic Burden of a Sustained Increase in Thyroid Cancer Incidence , 2013, Cancer Epidemiology, Biomarkers & Prevention.

[9]  Pedro Antonio Gutiérrez,et al.  Ordinal Classification Using Hybrid Artificial Neural Networks with Projection and Kernel Basis Functions , 2012, HAIS.

[10]  Brendan J. Frey,et al.  Classifying and segmenting microscopy images with deep multiple instance learning , 2015, Bioinform..

[11]  Guy Cazuguel,et al.  Multiple-Instance Learning for Medical Image and Video Analysis , 2017, IEEE Reviews in Biomedical Engineering.

[12]  E. Cibas,et al.  The Bethesda System for Reporting Thyroid Cytopathology , 2009, Springer International Publishing.

[13]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[14]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[15]  Paul Babyn,et al.  Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network , 2017, Journal of Digital Imaging.

[16]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Panagiota Spyridonos,et al.  Design of a multi-classifier system for discriminating benign from malignant thyroid nodules using routinely H&E-stained cytological images , 2008, Comput. Biol. Medicine.

[18]  B. Gopinath,et al.  Computer-aided diagnosis system for classifying benign and malignant thyroid nodules in multi-stained FNAB cytological images , 2013, Australasian Physical & Engineering Sciences in Medicine.

[19]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  J. Jonklaas,et al.  Thyroid nodules. , 2012, The Medical clinics of North America.

[21]  Edward Kim,et al.  A deep semantic mobile application for thyroid cytopathology , 2016, SPIE Medical Imaging.

[22]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[23]  Pranab Dey,et al.  Artificial neural network model to distinguish follicular adenoma from follicular carcinoma on fine needle aspiration of thyroid , 2018, Diagnostic cytopathology.

[24]  Dimitrios Koutsouris,et al.  Artificial Neural Networks as Decision Support Tools in Cytopathology: Past, Present, and Future , 2016, Biomedical engineering and computational biology.

[25]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[26]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[27]  Claire W Michael,et al.  Group consensus review minimizes the diagnosis of “follicular lesion of undetermined significance” and improves cytohistologic concordance , 2012, Diagnostic cytopathology.

[28]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[29]  Ruchi Srivastava,et al.  Implementation of the bethesda system for reporting thyroid cytopathology: Interobserver concordance and reclassification of previously inconclusive aspirates , 2014, Diagnostic cytopathology.

[30]  Pedro Antonio Gutiérrez,et al.  Ordinal Regression Methods: Survey and Experimental Study , 2016, IEEE Transactions on Knowledge and Data Engineering.

[31]  Tanushri Mukherjee,et al.  Artificial Intelligence in Cytopathology: A Neural Network to Identify Papillary Carcinoma on Thyroid Fine-Needle Aspiration Cytology Smears , 2018, Journal of pathology informatics.

[32]  Jing Yu,et al.  Classification of thyroid nodules in ultrasound images using deep model based transfer learning and hybrid features , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[34]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[35]  K. Aldape,et al.  Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care , 2017, npj Precision Oncology.

[36]  Edmond Sabo,et al.  Computerized cytometry and wavelet analysis of follicular lesions for detecting malignancy: A pilot study in thyroid cytology , 2017, Surgery.

[37]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[38]  Aimin Hao,et al.  Multitask Cascade Convolution Neural Networks for Automatic Thyroid Nodule Detection and Recognition , 2019, IEEE Journal of Biomedical and Health Informatics.

[39]  Weiwei Liu,et al.  An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images , 2018, Scientific Reports.