Micro‐Net: A unified model for segmentation of various objects in microscopy images

HighlightsA unified deep learning framework for segmentation of objects (cell nuclei, cells, and multi‐cellular objects such as glandular structures) in two main types of microscopy images: fluorescence and histology.The proposed Micro‐Net is aimed at better object localization in the face of varying intensities and is robust to noise.Detailed experimentation & comparative evaluation on publicly available data sets and a new image dataset that is made public with this paper.Demonstration of robustness of the algorithm to high levels of noise. Graphical abstract Figure. No caption available. ABSTRACT Object segmentation and structure localization are important steps in automated image analysis pipelines for microscopy images. We present a convolution neural network (CNN) based deep learning architecture for segmentation of objects in microscopy images. The proposed network can be used to segment cells, nuclei and glands in fluorescence microscopy and histology images after slight tuning of input parameters. The network trains at multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi‐resolution deconvolution filters. The extra convolutional layers which bypass the max‐pooling operation allow the network to train for variable input intensities and object size and make it robust to noisy data. We compare our results on publicly available data sets and show that the proposed network outperforms recent deep learning algorithms.

[1]  David B. A. Epstein,et al.  Tumor Segmentation in Whole Slide Images Using Persistent Homology and Deep Convolutional Features , 2017, MIUA.

[2]  Anant Madabhushi,et al.  Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[3]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Xudong Jiang,et al.  Accurate Cervical Cell Segmentation from Overlapping Clumps in Pap Smear Images , 2017, IEEE Trans. Medical Imaging.

[5]  A. Huisman,et al.  Automatic Nuclei Segmentation in H&E Stained Breast Cancer Histopathology Images , 2013, PloS one.

[6]  Yang Li,et al.  Gland Instance Segmentation by Deep Multichannel Side Supervision , 2016, MICCAI.

[7]  Karl Rohr,et al.  Efficient globally optimal segmentation of cells in fluorescence microscopy images using level sets and convex energy functionals , 2012, Medical Image Anal..

[8]  Yang Li,et al.  Gland Instance Segmentation Using Deep Multichannel Neural Networks , 2016, IEEE Transactions on Biomedical Engineering.

[9]  Anindya Sarkar,et al.  Structure and Context in Prostatic Gland Segmentation and Classification , 2012, MICCAI.

[10]  F. Markowetz,et al.  Quantitative Image Analysis of Cellular Heterogeneity in Breast Tumors Complements Genomic Profiling , 2012, Science Translational Medicine.

[11]  Ehud Rivlin,et al.  Memory based active contour algorithm using pixel-level classified images for colon crypt segmentation , 2015, Comput. Medical Imaging Graph..

[12]  Cenk Sokmensuer,et al.  Automatic segmentation of colon glands using object-graphs , 2010, Medical Image Anal..

[13]  G. Li,et al.  A novel multitarget tracking algorithm for Myosin VI protein molecules on actin filaments in TIRFM sequences , 2015, Journal of microscopy.

[14]  Erik H. W. Meijering,et al.  Cell Segmentation: 50 Years Down the Road [Life Sciences] , 2012, IEEE Signal Processing Magazine.

[15]  Nasir M. Rajpoot,et al.  A Stochastic Polygons Model for Glandular Structures in Colon Histology Images , 2015, IEEE Transactions on Medical Imaging.

[16]  Nasir M. Rajpoot,et al.  Multi‐resolution cell orientation congruence descriptors for epithelium segmentation in endometrial histology images , 2017, Medical Image Anal..

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Brendan J. Frey,et al.  Classifying and segmenting microscopy images with deep multiple instance learning , 2015, Bioinform..

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  H. Wu,et al.  Segmentation of intestinal gland images with iterative region growing , 2005, Journal of microscopy.

[21]  Fabian Rudolf,et al.  Accurate cell segmentation in microscopy images using membrane patterns , 2014, Bioinform..

[22]  Ghassan Hamarneh,et al.  Local Optimization Based Segmentation of Spatially-Recurring, Multi-Region Objects With Part Configuration Constraints , 2014, IEEE Transactions on Medical Imaging.

[23]  E. Meijering Cell Segmentation : 50 Years Down the Road , 2012 .

[24]  David B. A. Epstein,et al.  RAMTaB: Robust Alignment of Multi-Tag Bioimages , 2012, PloS one.

[25]  Hao Chen,et al.  DCAN: Deep contour‐aware networks for object instance segmentation from histology images , 2017, Medical Image Anal..

[26]  David B. A. Epstein,et al.  MIMONet: Gland Segmentation Using Multi-Input-Multi-Output Convolutional Neural Network , 2017, MIUA.

[27]  Hao Chen,et al.  Gland segmentation in colon histology images: The glas challenge contest , 2016, Medical Image Anal..

[28]  Nasir M. Rajpoot,et al.  SAMS-NET: Stain-aware multi-scale network for instance-based nuclei segmentation in histology images , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[29]  W. Schubert,et al.  Analyzing proteome topology and function by automated multidimensional fluorescence microscopy , 2006, Nature Biotechnology.

[30]  David B. A. Epstein,et al.  Robust normalization protocols for multiplexed fluorescence bioimage analysis , 2016, BioData Mining.

[31]  Stephen J. McKenna,et al.  Structure Prediction for Gland Segmentation With Hand-Crafted and Deep Convolutional Features , 2018, IEEE Transactions on Medical Imaging.

[32]  R. A. Zoroofi,et al.  An image analysis approach for automatic malignancy determination of prostate pathological images , 2007, Cytometry. Part B, Clinical cytometry.

[33]  Xiaobo Zhou,et al.  Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[34]  Carolina Wählby,et al.  Automated Training of Deep Convolutional Neural Networks for Cell Segmentation , 2017, Scientific Reports.

[35]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[36]  David B. A. Epstein,et al.  MIMO-Net: A multi-input multi-output convolutional neural network for cell segmentation in fluorescence microscopy images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[37]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[38]  David B. A. Epstein,et al.  Glandular Morphometrics for Objective Grading of Colorectal Adenocarcinoma Histology Images , 2017, Scientific Reports.