Unsupervised Learning for Cell-Level Visual Representation in Histopathology Images With Generative Adversarial Networks

The visual attributes of cells, such as the nuclear morphology and chromatin openness, are critical for histopathology image analysis. By learning cell-level visual representation, we can obtain a rich mix of features that are highly reusable for various tasks, such as cell-level classification, nuclei segmentation, and cell counting. In this paper, we propose a unified generative adversarial networks architecture with a new formulation of loss to perform robust cell-level visual representation learning in an unsupervised setting. Our model is not only label-free and easily trained but also capable of cell-level unsupervised classification with interpretable visualization, which achieves promising results in the unsupervised classification of bone marrow cellular components. Based on the proposed cell-level visual representation learning, we further develop a pipeline that exploits the varieties of cellular elements to perform histopathology image classification, the advantages of which are demonstrated on bone marrow datasets.

[1]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jianzhong Wu,et al.  Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[3]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[4]  Joel H. Saltz,et al.  Automatic histopathology image analysis with CNNs , 2016, 2016 New York Scientific Data Summit (NYSDS).

[5]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[6]  Carlos López,et al.  Evaluation of cytokeratin-19 in breast cancer tissue samples: a comparison of automatic and manual evaluations of scanned tissue microarray cylinders , 2015, IWBBIO.

[7]  Yan Xu,et al.  Deep learning of feature representation with multiple instance learning for medical image analysis , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[9]  Hai Su,et al.  Automatic Myonuclear Detection in Isolated Single Muscle Fibers Using Robust Ellipse Fitting and Sparse Representation , 2014, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[10]  Cecilia Di Ruberto,et al.  Leucocyte classification for leukaemia detection using image processing techniques , 2014, Artif. Intell. Medicine.

[11]  Jun Xu,et al.  Fusing Heterogeneous Features From Stacked Sparse Autoencoder for Histopathological Image Analysis , 2016, IEEE Journal of Biomedical and Health Informatics.

[12]  Leyza Baldo Dorini,et al.  Semiautomatic White Blood Cell Segmentation Based on Multiscale Analysis , 2013, IEEE Journal of Biomedical and Health Informatics.

[13]  Fabio A. González,et al.  A Deep Learning Architecture for Image Representation, Visual Interpretability and Automated Basal-Cell Carcinoma Cancer Detection , 2013, MICCAI.

[14]  Wiro J. Niessen,et al.  Advanced Level-Set-Based Cell Tracking in Time-Lapse Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[15]  Wei Liu,et al.  Towards Large-Scale Histopathological Image Analysis: Hashing-Based Image Retrieval , 2015, IEEE Transactions on Medical Imaging.

[16]  H. Gralnick,et al.  Proposals for the Classification of the Acute Leukaemias French‐American‐British (FAB) Co‐operative Group , 1976, British journal of haematology.

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  H. Irshad,et al.  Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential , 2014, IEEE Reviews in Biomedical Engineering.

[19]  Murray Shanahan,et al.  Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders , 2016, ArXiv.

[20]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[21]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[22]  Hai Su,et al.  Weighted Hashing with Multiple Cues for Cell-Level Analysis of Histopathological Images , 2015, IPMI.

[23]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[24]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[25]  Mu-Chun Su,et al.  A Neural-Network-Based Approach to White Blood Cell Classification , 2014, TheScientificWorldJournal.

[26]  Anant Madabhushi,et al.  A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images , 2016, Neurocomputing.

[27]  Hai Su,et al.  Cell Encoding for Histopathology Image Classification , 2017, MICCAI.

[28]  Feng Lin,et al.  Dual Spatial Pyramid On Rotation Invariant Texture Feature For HEp-2 Cell Classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[29]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[30]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[31]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[32]  Anil K. Jain,et al.  Prostate cancer detection: Fusion of cytological and textural features , 2011, Journal of pathology informatics.

[33]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[34]  Oliver Schmitt,et al.  Morphological multiscale decomposition of connected regions with emphasis on cell clusters , 2009, Comput. Vis. Image Underst..

[35]  Daniel Heim,et al.  Detection and Segmentation of Cell Nuclei in Virtual Microscopy Images: A Minimum-Model Approach , 2012, Scientific Reports.

[36]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[37]  Cigdem Demir,et al.  Attributed Relational Graphs for Cell Nucleus Segmentation in Fluorescence Microscopy Images , 2013, IEEE Transactions on Medical Imaging.

[38]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[39]  Jost Tobias Springenberg,et al.  Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks , 2015, ICLR.

[40]  Oscar Déniz-Suárez,et al.  A parallel solution for high resolution histological image analysis , 2012, Comput. Methods Programs Biomed..

[41]  Osman Kalender,et al.  Automatic segmentation, counting, size determination and classification of white blood cells , 2014 .

[42]  Dwarikanath Mahapatra,et al.  Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis , 2017, MICCAI.

[43]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[44]  Vincent Lepetit,et al.  You Should Use Regression to Detect Cells , 2015, MICCAI.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  Rong-Ming Chen,et al.  Blood Cell Image Classification Based on Hierarchical SVM , 2011, 2011 IEEE International Symposium on Multimedia.

[47]  Bahram Parvin,et al.  Invariant Delineation of Nuclear Architecture in Glioblastoma Multiforme for Clinical and Molecular Association , 2013, IEEE Transactions on Medical Imaging.

[48]  E. Myers,et al.  A 3D Digital Atlas of C. elegans and Its Application To Single-Cell Analyses , 2009, Nature Methods.

[49]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[50]  Ajoy Kumar Ray,et al.  Hybrid segmentation, characterization and classification of basal cell nuclei from histopathological images of normal oral mucosa and oral submucous fibrosis , 2012, Expert Syst. Appl..

[51]  Jianwei Zhao,et al.  Automatic detection and classification of leukocytes using convolutional neural networks , 2016, Medical & Biological Engineering & Computing.

[52]  Hai Su,et al.  High-throughput histopathological image analysis via robust cell segmentation and hashing , 2015, Medical Image Anal..

[53]  Ting Chen,et al.  Deep Learning Based Automatic Immune Cell Detection for Immunohistochemistry Images , 2014, MLMI.

[54]  Huachun Tan,et al.  Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering , 2016, IJCAI.

[55]  Fang Zhang,et al.  Deep convolutional activation features for large scale Brain Tumor histopathology image classification and segmentation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[58]  Zhipeng Jia,et al.  Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features , 2017, BMC Bioinformatics.

[59]  Dipti Patra,et al.  An ensemble classifier system for early diagnosis of acute lymphoblastic leukemia in blood microscopic images , 2013, Neural Computing and Applications.

[60]  Metin Nafi Gürcan,et al.  Computerized classification of intraductal breast lesions using histopathological images , 2011, IEEE Transactions on Biomedical Engineering.

[61]  Eric Cosatto,et al.  Classification of mitotic figures with convolutional neural networks and seeded blob features , 2013, Journal of pathology informatics.

[62]  Juan V. Lorenzo-Ginori,et al.  Cervical Cell Classification Using Features Related to Morphometry and Texture of Nuclei , 2013, CIARP.

[63]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[64]  Jinhua Yu,et al.  Reconstruction of Thin-Slice Medical Images Using Generative Adversarial Network , 2017, MLMI@MICCAI.