Assisting the examination of large histopathological slides with adaptive forests

&NA; The examination of biopsy samples plays a central role in the diagnosis and staging of numerous diseases, including most cancer types. However, because of the large size of the acquired images, the localization and quantification of diseased portions of a tissue is usually time‐consuming, as pathologists must scroll through the whole slide to look for objects of interest which are often only scarcely distributed. In this work, we introduce an approach to facilitate the visual inspection of large digital histopathological slides. Our method builds on a random forest classifier trained to segment the structures sought by the pathologist. However, moving beyond the pixelwise segmentation task, our main contribution is an interactive exploration framework including: (i) a region scoring function which is used to rank and sequentially display regions of interest to the user, and (ii) a relevance feedback capability which leverages human annotations collected on each suggested region. Thereby, an online domain adaptation of the learned pixelwise segmentation model is performed, so that the region scores adapt on‐the‐fly to possible discrepancies between the original training data and the slide at hand. Three real‐time update strategies are compared, including a novel approach based on online gradient descent which supports faster user interaction than an accurate delineation of objects. Our method is evaluated on the task of extramedullary hematopoiesis quantification within mouse liver slides. We assess quantitatively the retrieval abilities of our approach and the benefit of the interactive adaptation scheme. Moreover, we demonstrate the possibility of extrapolating, after a partial exploration of the slide, the surface covered by hematopoietic cells within the whole tissue. HighlightsInteractive framework to efficiently explore large histopathological slides.Online adaptation from user inputs to adjust the model to new conditions.Application to hematopoiesis quantification within mouse liver samples.Inference of whole‐slide quantification from a partial exploration.Compatibility with one‐click user inputs for lighter interaction. Graphical abstract Figure. No caption available.

[1]  Horst Bischof,et al.  On-line Random Forests , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  Hai Su,et al.  Robust Cell Detection and Segmentation in Histopathological Images Using Sparse Reconstruction and Stacked Denoising Autoencoders , 2015, MICCAI.

[3]  Isabelle Bloch,et al.  Multi-organ Localization Combining Global-to-Local Regression and Confidence Maps , 2014, MICCAI.

[4]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[5]  Nassir Navab,et al.  Scale-Adaptive Forest Training via an Efficient Feature Sampling Scheme , 2015, MICCAI.

[6]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[7]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[8]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[9]  Antonio Criminisi,et al.  Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2012, Found. Trends Comput. Graph. Vis..

[10]  Mechthild Krause,et al.  CD8+ tumour‐infiltrating lymphocytes in relation to HPV status and clinical outcome in patients with head and neck cancer after postoperative chemoradiotherapy: A multicentre study of the German cancer consortium radiation oncology group (DKTK‐ROG) , 2016, International journal of cancer.

[11]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[12]  Ben Glocker,et al.  Encoding atlases by randomized classification forests for efficient multi-atlas label propagation , 2014, Medical Image Anal..

[13]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[14]  Anindya Sarkar,et al.  Prostate Cancer Grading: Use of Graph Cut and Spatial Arrangement of Nuclei , 2014, IEEE Transactions on Medical Imaging.

[15]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[16]  Antonio Criminisi,et al.  Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes , 2009 .

[17]  Nicolas Loménie,et al.  Time-efficient sparse analysis of histopathological whole slide images , 2011, Comput. Medical Imaging Graph..

[18]  Steven C. H. Hoi,et al.  OTL: A Framework of Online Transfer Learning , 2010, ICML.

[19]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[20]  Antonio Criminisi,et al.  Fast Multiple Organ Detection and Localization in Whole-Body MR Dixon Sequences , 2011, MICCAI.

[21]  Saskia Le Cessie,et al.  Assessment of Interobserver Variability and Histologic Parameters to Improve Reliability in Classification and Grading of Central Cartilaginous Tumors , 2009, The American journal of surgical pathology.

[22]  Bernd Fischer,et al.  CellCognition: time-resolved phenotype annotation in high-throughput live cell imaging , 2010, Nature Methods.

[23]  Navid Farahani,et al.  whole slide imaging in pathology: advantages, limitations, and emerging perspectives , 2015 .

[24]  C. Acar,et al.  Comparison of 1998 WHO/ISUP and 1973 WHO Classifications for Interobserver Variability in Grading of Papillary Urothelial Neoplasms of the Bladder , 2007, Urologia Internationalis.

[25]  Linda G. Shapiro,et al.  Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: a Comparative Study , 2014, 2014 22nd International Conference on Pattern Recognition.

[26]  Erik G. Learned-Miller,et al.  Online domain adaptation of a pre-trained cascade of classifiers , 2011, CVPR 2011.

[27]  Horst K. Hahn,et al.  Practical quantification of necrosis in histological whole-slide images , 2013, Comput. Medical Imaging Graph..

[28]  Yukako Yagi,et al.  Staining Correction in Digital Pathology by Utilizing a Dye Amount Table , 2015, Journal of Digital Imaging.

[29]  Hai Su,et al.  Robust Cell Detection and Segmentation in Histopathological Images Using Sparse Reconstruction and Stacked Denoising Autoencoders , 2015, MICCAI.

[30]  Claus Bahlmann,et al.  Computer-aided gleason grading of prostate cancer histopathological images using texton forests , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[31]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[32]  Devrim Onder,et al.  A Review on Color Normalization and Color Deconvolution Methods in Histopathology , 2014, Applied immunohistochemistry & molecular morphology : AIMM.

[33]  Joseph,et al.  Imagable 4 T 1 model for the study of late stage breast cancer , 2008 .

[34]  Serge J. Belongie,et al.  Unsupervised Color Decomposition Of Histologically Stained Tissue Samples , 2003, NIPS.

[35]  Barbara Caputo,et al.  Leveraging over prior knowledge for online learning of visual categories , 2012, BMVC.

[36]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[37]  Max A. Viergever,et al.  Breast Cancer Histopathology Image Analysis: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[38]  Nassir Navab,et al.  Structure-preserved color normalization for histological images , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[39]  Anil V Parwani,et al.  Clinical examination and validation of primary diagnosis in anatomic pathology using whole slide digital images. , 2011, Archives of pathology & laboratory medicine.

[40]  Nassir Navab,et al.  Leveraging Random Forests for Interactive Exploration of Large Histological Images , 2014, MICCAI.

[41]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[42]  Ian F. Pollack,et al.  Pathologist Interobserver Variability of Histologic Features in Childhood Brain Tumors: Results from the CCG-945 Study , 2008, Pediatric and developmental pathology : the official journal of the Society for Pediatric Pathology and the Paediatric Pathology Society.

[43]  Hui Kong,et al.  Partitioning Histopathological Images: An Integrated Framework for Supervised Color-Texture Segmentation and Cell Splitting , 2011, IEEE Transactions on Medical Imaging.

[44]  Ludwig Kappos,et al.  Quantifying Progression of Multiple Sclerosis via Classification of Depth Videos , 2014, MICCAI.

[45]  Nasir Rajpoot,et al.  A Non-Linear Mapping Approach to Stain Normalisation in Digital Histopathology Images using Image-Specific Colour Deconvolution , 2014 .

[46]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[47]  Nassir Navab,et al.  Learning from Multiple Experts with Random Forests: Application to the Segmentation of the Midbrain in 3D Ultrasound , 2013, MICCAI.

[48]  Walter H Henricks,et al.  Validation of whole slide imaging for primary diagnosis in surgical pathology. , 2013, Archives of pathology & laboratory medicine.

[49]  Horst K. Hahn,et al.  A comparison of sampling strategies for histological image analysis , 2011, Journal of pathology informatics.

[50]  Shaimaa Al-Janabi,et al.  Digital pathology: current status and future perspectives , 2012, Histopathology.

[51]  J. Russo,et al.  Breast carcinoma malignancy grading by Bloom–Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index , 2005, Modern Pathology.

[52]  Anant Madabhushi,et al.  A Boosted Bayesian Multiresolution Classifier for Prostate Cancer Detection From Digitized Needle Biopsies , 2012, IEEE Transactions on Biomedical Engineering.

[53]  Ullrich Köthe,et al.  Learning to count with regression forest and structured labels , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[54]  Jun Kong,et al.  Computer-aided prognosis of neuroblastoma on whole-slide images: Classification of stromal development , 2009, Pattern Recognit..

[55]  Aaron Fenster,et al.  Prostate Histopathology: Learning Tissue Component Histograms for Cancer Detection and Classification , 2013, IEEE Transactions on Medical Imaging.

[56]  Joseph,et al.  Imagable 4T1 model for the study of late stage breast cancer , 2008, BMC Cancer.

[57]  Yee Whye Teh,et al.  Mondrian Forests: Efficient Online Random Forests , 2014, NIPS.

[58]  Claus Bahlmann,et al.  Automated detection of diagnostically relevant regions in H&E stained digital pathology slides , 2012, Medical Imaging.

[59]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[60]  Abderrahim Elmoataz,et al.  Multi-resolution graph-based analysis of histopathological whole slide images: Application to mitotic cell extraction and visualization , 2011, Comput. Medical Imaging Graph..

[61]  H. Jarodzka,et al.  Expertise under the microscope: processing histopathological slides , 2014, Medical education.

[62]  Jun Kong,et al.  Digital Pathology: Data-Intensive Frontier in Medical Imaging , 2012, Proceedings of the IEEE.

[63]  Horst Bischof,et al.  Towards Automatic Bone Age Estimation from MRI: Localization of 3D Anatomical Landmarks , 2014, MICCAI.

[64]  Dimitris N. Metaxas,et al.  Entangled Decision Forests and Their Application for Semantic Segmentation of CT Images , 2011, IPMI.