Histogram of Cell Types: Deep Learning for Automated Bone Marrow Cytology

Bone marrow cytology is required to make a hematological diagnosis, influencing critical clinical decision points in hematology. However, bone marrow cytology is tedious, limited to experienced reference centers and associated with high inter-observer variability. This may lead to a delayed or incorrect diagnosis, leaving an unmet need for innovative supporting technologies. We have developed the first ever end-to-end deep learningbased technology for automated bone marrow cytology. Starting with a bone marrow aspirate digital whole slide image, our technology rapidly and automatically detects suitable regions for cytology, and subsequently identifies and classifies all bone marrow cells in each region. This collective cytomorphological information is captured in a novel representation called Histogram of Cell Types (HCT) quantifying bone marrow cell class probability distribution and acting as a cytological “patient fingerprint”. The approach achieves high accuracy in region detection (0.97 accuracy and 0.99 ROC AUC), and cell detection and cell classification (0.75 mAP, 0.78 F1-score, Log-average miss rate of 0.31). HCT has potential to revolutionize hematopathology diagnostic workflows, leading to more cost-effective, accurate diagnosis and opening the door to precision medicine.

[1]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[2]  Diganta Misra.  Mish: A Self Regularized Non-Monotonic Activation Function , 2020, BMVC.

[3]  Jeffrey A. Golden,et al.  Deep Learning Algorithms for Detection of Lymph Node Metastases From Breast Cancer: Helping Artificial Intelligence Be Seen. , 2017, JAMA.

[4]  Jun-Wei Hsieh,et al.  CSPNet: A New Backbone that can Enhance Learning Capability of CNN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Liron Pantanowitz,et al.  Challenges in the Development, Deployment & Regulation of Artificial Intelligence (AI) in Anatomical Pathology. , 2020, The American journal of pathology.

[6]  Hong Liu,et al.  Bone Marrow Cells Detection: A Technique for the Microscopic Image Analysis , 2018, Journal of Medical Systems.

[7]  D. Steensma.  Myelodysplastic syndromes current treatment algorithm 2018 , 2018, Blood Cancer Journal.

[8]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[9]  P. Font,et al.  Inter-observer variance with the diagnosis of myelodysplastic syndromes (MDS) following the 2008 WHO classification , 2012, Annals of Hematology.

[10]  David A. Gutman,et al.  Machine-Based Detection and Classification for Bone Marrow Aspirate Differential Counts: Initial Development Focusing on Non-Neoplastic Cells , 2019, Laboratory Investigation.

[11]  Andrew H. Beck,et al.  Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association , 2019, The Journal of pathology.

[12]  William R. Schwartz,et al.  A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[13]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[14]  W. Rodgers.  Bone marrow aspiration. , 2003, Orthopedics.

[15]  Hakim Ghazzai,et al.  Object Detection Learning Techniques for Autonomous Vehicle Applications , 2019, 2019 IEEE International Conference of Vehicular Electronics and Safety (ICVES).

[16]  Wei Fang,et al.  A novel YOLO-Based real-time people counting approach , 2017, 2017 International Smart Cities Conference (ISC2).

[17]  Quoc V. Le,et al.  DropBlock: A regularization method for convolutional networks , 2018, NeurIPS.

[18]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[19]  A. Porwit,et al.  ICSH guidelines for the standardization of bone marrow specimens and reports , 2008, International journal of laboratory hematology.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Bernard Têtu,et al.  Canadian licensure for the use of digital pathology for routine diagnoses: one more step toward a new era of pathology practice without borders. , 2014, Archives of pathology & laboratory medicine.

[23]  Zhaohui Zheng,et al.  Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression , 2019, AAAI.

[24]  F. Ravandi,et al.  Evaluating measurable residual disease in acute myeloid leukemia. , 2018, Blood advances.

[25]  Jin Woo Choi,et al.  White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks , 2017, PloS one.

[26]  Bob Löwenberg,et al.  Diagnosis and management of AML in adults: 2017 ELN recommendations from an international expert panel. , 2017, Blood.

[27]  Sa A Wang,et al.  Initial Diagnostic Workup of Acute Leukemia: Guideline From the College of American Pathologists and the American Society of Hematology. , 2017, Archives of pathology & laboratory medicine.

[28]  Giovanni M. Lujan,et al.  Dissecting the Business Case for Adoption and Implementation of Digital Pathology: A White Paper from the Digital Pathology Association , 2021, Journal of pathology informatics.

[29]  Ellery Wulczyn,et al.  Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer , 2018, npj Digital Medicine.

[30]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.

[33]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[34]  N. Radakovich,et al.  Artificial Intelligence in Hematology: Current Challenges and Opportunities , 2020, Current Hematologic Malignancy Reports.

[35]  Saeed Hassanpour,et al.  Deep Learning for Classification of Colorectal Polyps on Whole-slide Images , 2017, Journal of pathology informatics.

[36]  Adrien Depeursinge,et al.  Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles , 2016, Medical Image Anal..

[37]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[38]  L. Pantanowitz,et al.  Quantitative Image Analysis for Tissue Biomarker Use: A White Paper From the Digital Pathology Association , 2021, Applied immunohistochemistry & molecular morphology : AIMM.

[39]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[40]  Ali N. Hasan,et al.  Deep Learning in Object Detection: a Review , 2020, 2020 International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (icABCD).

[41]  Tae-Yeong Kwak,et al.  Artificial Intelligence in Pathology , 2018, Journal of pathology and translational medicine.

[42]  M. Gurcan,et al.  Digital pathology and artificial intelligence. , 2019, The Lancet. Oncology.

[43]  Linda A Bradley,et al.  Principles of analytic validation of immunohistochemical assays: Guideline from the College of American Pathologists Pathology and Laboratory Quality Center. , 2014, Archives of pathology & laboratory medicine.

[44]  Z. Estrov,et al.  Implications of discrepancy in morphologic diagnosis of myelodysplastic syndrome between referral and tertiary care centers. , 2011, Blood.

[45]  Y. Yatomi,et al.  Inter-observer variance and the need for standardization in the morphological classification of myelodysplastic syndrome. , 2018, Leukemia research.

[46]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Alexis B. Carter,et al.  Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. , 2013, Archives of pathology & laboratory medicine.

[48]  Westley Knight.  The Path Ahead , 2019 .