Cell Segmentation Using a Similarity Interface With a Multi-Task Convolutional Neural Network

Even though convolutional neural networks (CNN) have been used for cell segmentation, they require pixel-level ground truth annotations. This paper proposes a multitask learning algorithm for cell detection and segmentation using CNNs. We use dot annotations placed inside each cell indicating approximate cell centroids to create training datasets for the detection and segmentation tasks. The segmentation task is used to map the input image to foreground versus background regions, whereas the detection task is used to predict the centroids of the cells. Our multitask model shares convolutional layers between the two tasks, while having task-specific output layers. Learning two tasks simultaneously reduces the risks of overfitting and also helps in separating overlapping cells better. We also introduce a similarity interface (SI) that can be integrated with our multitask network to allow easy adaptation between domains, and to compensate for the variability in contrast and texture of cells seen in microscopy images. The SI comprises an unsupervised first layer in combination with a neighborhood similarity layer (NSL). A layer of logistic sigmoid functions is used as an unsupervised first layer to separate clustered image patches from each other. The NSL transforms its input feature map at a given pixel by computing its similarity to the surrounding neighborhood. Our proposed method achieves higher/comparable detection and segmentation scores as compared to recent state-of-the-art methods with significantly reduced effort for generating training data.

[1]  Hai Su,et al.  Beyond Classification: Structured Regression for Robust Cell Detection Using Convolutional Neural Network , 2015, MICCAI.

[2]  Andrew Zisserman,et al.  Multi-Task Convolutional Neural Network for Patient Detection and Skin Segmentation in Continuous Non-Contact Vital Sign Monitoring , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[3]  Serge Beucher,et al.  Use of watersheds in contour detection , 1979 .

[4]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Lin Yang,et al.  Robust Muscle Cell Quantification Using Structured Edge Detection and Hierarchical Segmentation , 2015, MICCAI.

[6]  Erik H. W. Meijering,et al.  Cell Segmentation: 50 Years Down the Road [Life Sciences] , 2012, IEEE Signal Processing Magazine.

[7]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[8]  Trevor Darrell,et al.  FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation , 2016, ArXiv.

[9]  Nilanjan Ray,et al.  Cell Detection with Deep Convolutional Neural Network and Compressed Sensing , 2017, ArXiv.

[10]  Ting Liu,et al.  Image Segmentation Using Hierarchical Merge Tree , 2015, IEEE Transactions on Image Processing.

[11]  Juho Kannala,et al.  Cell Tracking via Proposal Generation and Selection , 2017, ArXiv.

[12]  Kannappan Palaniappan,et al.  Cell Segmentation Using Coupled Level Sets and Graph-Vertex Coloring , 2006, MICCAI.

[13]  Chen Huang,et al.  Local Similarity-Aware Deep Feature Embedding , 2016, NIPS.

[14]  Zhang Yi,et al.  Cell tracking using deep neural networks with multi-task learning , 2017, Image Vis. Comput..

[15]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[16]  Pascal Fua,et al.  Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences , 2017, IEEE Transactions on Medical Imaging.

[17]  Carolina Wählby,et al.  Feature Augmented Deep Neural Networks for Segmentation of Cells , 2016, ECCV Workshops.

[18]  Julian Yarkony,et al.  Cell Detection and Segmentation Using Correlation Clustering , 2014, MICCAI.

[19]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[20]  Dwi Anoraganingrum,et al.  Cell segmentation with median filter and mathematical morphology operation , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[21]  Tolga Tasdizen,et al.  Domain adaptation for biomedical image segmentation using adversarial training , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[22]  Tolga Tasdizen,et al.  Improving the robustness of convolutional networks to appearance variability in biomedical images , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[23]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[24]  Joakim Jalden,et al.  Global Linking of Cell Tracks Using the Viterbi Algorithm , 2015, IEEE Transactions on Medical Imaging.

[25]  Nilanjan Ray,et al.  Cell Counting by Regression Using Convolutional Neural Network , 2016, ECCV Workshops.

[26]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[28]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[29]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[30]  Müjdat Çetin,et al.  Disjunctive normal shape models , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[31]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[32]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Shuiwang Ji,et al.  Residual Deconvolutional Networks for Brain Electron Microscopy Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[34]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[35]  Andrew Zisserman,et al.  Microscopy cell counting with fully convolutional regression networks , 2015 .

[36]  Andrew G. Dempster,et al.  Segmentation of blood images using morphological operators , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[37]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[38]  Yinhui Zhang,et al.  Pedestrian Detection Aided by Deep Learning Attributes Task , 2016, CCPR.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[41]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[42]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[43]  Fred A. Hamprecht,et al.  Learning to Segment: Training Hierarchical Segmentation under a Topological Loss , 2015, MICCAI.

[44]  Lin Yang,et al.  A Novel Cell Detection Method Using Deep Convolutional Neural Network and Maximum-Weight Independent Set , 2015, MICCAI.

[45]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[46]  Andrew Zisserman,et al.  Learning to Detect Cells Using Non-overlapping Extremal Regions , 2012, MICCAI.

[47]  Ting Liu,et al.  SSHMT: Semi-supervised Hierarchical Merge Tree for Electron Microscopy Image Segmentation , 2016, ECCV.

[48]  Peng Wang,et al.  Semantic Instance Segmentation via Deep Metric Learning , 2017, ArXiv.

[49]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[50]  Yu Zhang,et al.  A Survey on Multi-Task Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.