Deep Voting and Structured Regression for Microscopy Image Analysis

Abstract Robust and accurate nuclei localization in microscopy images can provide crucial clues for accurate computer-aided diagnosis. In this chapter, we present two methods that rely on convolutional neural networks (CNNs) to solve this problem. The first one is named as deep voting, which is a CNN based hough voting method used to localize nucleus centroids that exhibit heavy cluttering and morphological variations. It mainly consists of the following two parts: (i) Given an input image, this model maps every local testing image patch to the proposed target information, which consists of several pairs of voting offset vectors and voting confidence. Voting offset vectors are used to specify the pixel coordinates each local testing image patch votes to, and the corresponding voting confidence is used as weight assigned to each vote. (ii) We collect the weighted votes from all the testing image patches and compute the final voting density map for the entire testing image in a way similar to Parzen-window estimation. The final nucleus positions are then identified by searching the local maxima of the density map. The second one is a novel CNN based structured regression model, which is shown to be able to handle touching cells, inhomogeneous background noises, and large variations in sizes and shapes. Given an input image patch, instead of providing a single class label like many traditional methods, it will generate the structured outputs (referred to as proximity patches). These proximity patches, which exhibit higher values for pixels near cell centers, will then be gathered from all testing image patches and fused to obtain the final proximity map, where the maximum positions indicate the cell centroids. Both methods only require a few training images with weak annotations (just one click near the center of the object). Experimental results demonstrate that the proposed method achieves significantly improved performance compared to the state-of-the-art methods and shows strong robustness when dealing with the complex touching cells, weak staining, and fuzzy boundaries.

[1]  Hai Su,et al.  An Integrated Framework for Automatic Ki-67 Scoring in Pancreatic Neuroendocrine Tumor , 2013, MICCAI.

[2]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Tianli Yu,et al.  Kernelized structural SVM learning for supervised object segmentation , 2011, CVPR 2011.

[4]  Stella X. Yu,et al.  Finding dots: Segmentation as popping out regions from boundaries , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  Lin Yang,et al.  Robust Segmentation of Overlapping Cells in Histopathology Specimens Using Parallel Seed Detection and Repulsive Level Set , 2012, IEEE Transactions on Biomedical Engineering.

[7]  Dinggang Shen,et al.  Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.

[8]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[9]  Jürgen Schmidhuber,et al.  Multi-dimensional Recurrent Neural Networks , 2007, ICANN.

[10]  Luc Van Gool,et al.  A Hough transform-based voting framework for action recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Hui Kong,et al.  Automated detection of cells from immunohistochemically-stained tissues: application to Ki-67 nuclei staining , 2012, Medical Imaging.

[13]  Marcus Liwicki,et al.  Scene labeling with LSTM recurrent neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Giovanni Montana,et al.  Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks , 2015, ICPRAM 2015.

[15]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[16]  Horst Bischof,et al.  Hough Networks for Head Pose Estimation and Facial Feature Localization , 2014, BMVC.

[17]  Wenyin Liu,et al.  HEp-2 cell pattern classification with discriminative dictionary learning , 2014, Pattern Recognit..

[18]  Yousef Al-Kofahi,et al.  Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images , 2010, IEEE Transactions on Biomedical Engineering.

[19]  Hai Su,et al.  Beyond Classification: Structured Regression for Robust Cell Detection Using Convolutional Neural Network , 2015, MICCAI.

[20]  Andrew Zisserman,et al.  Learning to Detect Cells Using Non-overlapping Extremal Regions , 2012, MICCAI.

[21]  Hui Kong,et al.  Partitioning Histopathological Images: An Integrated Framework for Supervised Color-Texture Segmentation and Cell Splitting , 2011, IEEE Transactions on Medical Imaging.

[22]  Adel Hafiane,et al.  Fuzzy Clustering and Active Contours for Histopathology Image Segmentation and Nuclei Detection , 2008, ACIVS.

[23]  Peter Kontschieder,et al.  Structured class-labels in random forests for semantic image labelling , 2011, 2011 International Conference on Computer Vision.

[24]  Ana Maria Mendonça,et al.  Cell Nuclei and Cytoplasm Joint Segmentation Using the Sliding Band Filter , 2010, IEEE Transactions on Medical Imaging.

[25]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[26]  Sven Behnke,et al.  Hierarchical Neural Networks for Image Interpretation , 2003, Lecture Notes in Computer Science.

[27]  Christian Igel,et al.  Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network , 2013, MICCAI.

[28]  B. S. Manjunath,et al.  Automated tool for the detection of cell nuclei in digital microscopic images: application to retinal images. , 2006, Molecular vision.

[29]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[30]  Julian Yarkony,et al.  Cell Detection and Segmentation Using Correlation Clustering , 2014, MICCAI.

[31]  Stephen T. C. Wong,et al.  Detection of blob objects in microscopic zebrafish images based on gradient vector diffusion , 2007, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[32]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[33]  Qing Yang,et al.  Iterative Voting for Inference of Structural Saliency and Characterization of Subcellular Events , 2007, IEEE Transactions on Image Processing.

[34]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[35]  Hai Su,et al.  Automatic Myonuclear Detection in Isolated Single Muscle Fibers Using Robust Ellipse Fitting and Sparse Representation , 2014, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[36]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[37]  Lin Yang,et al.  Spatial Clockwork Recurrent Neural Network for Muscle Perimysium Segmentation , 2016, MICCAI.

[38]  Fabio A. González,et al.  A Deep Learning Architecture for Image Representation, Visual Interpretability and Automated Basal-Cell Carcinoma Cancer Detection , 2013, MICCAI.

[39]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[40]  B. Schiele,et al.  Combined Object Categorization and Segmentation With an Implicit Shape Model , 2004 .

[41]  Ronald M. Summers,et al.  A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations , 2014, MICCAI.

[42]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[43]  Hai Su,et al.  Deep Voting: A Robust Approach Toward Nucleus Localization in Microscopy Images , 2015, MICCAI.

[44]  Juergen Gall,et al.  Class-specific Hough forests for object detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Lin Yang,et al.  Automatic Image Analysis of Histopathology Specimens Using Concave Vertex Graph , 2008, MICCAI.

[46]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[48]  Luca Maria Gambardella,et al.  Fast image scanning with deep max-pooling convolutional neural networks , 2013, 2013 IEEE International Conference on Image Processing.

[49]  Lin Yang,et al.  An Automatic Learning-Based Framework for Robust Nucleus Segmentation , 2016, IEEE Transactions on Medical Imaging.

[50]  Shu Liao,et al.  Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images , 2013, MICCAI.