GRUU-Net: Integrated convolutional and gated recurrent neural network for cell segmentation

Cell segmentation in microscopy images is a common and challenging task. In recent years, deep neural networks achieved remarkable improvements in the field of computer vision. The dominant paradigm in segmentation is using convolutional neural networks, less common are recurrent neural networks. In this work, we propose a new deep learning method for cell segmentation, which integrates convolutional neural networks and gated recurrent neural networks over multiple image scales to exploit the strength of both types of networks. To increase the robustness of the training and improve segmentation, we introduce a novel focal loss function. We also present a distributed scheme for optimized training of the integrated neural network. We applied our proposed method to challenging data of glioblastoma cell nuclei and performed a quantitative comparison with state-of-the-art methods. Insights on how our extensions affect training and inference are also provided. Moreover, we benchmarked our method using a wide spectrum of all 22 real microscopy datasets of the Cell Tracking Challenge.

[1]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Dimitris N. Metaxas,et al.  Pixel-wise neural cell instance segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[3]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[4]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[5]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  한보형,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015 .

[7]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jiro Katto,et al.  Deep Residual Learning for Image Compression , 2019, CVPR Workshops.

[9]  Olaf Ronneberger,et al.  Cell segmentation and tracking in phase contrast images using graph cut with asymmetric boundary costs , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[10]  Juho Kannala,et al.  Cell Tracking via Proposal Generation and Selection , 2017, ArXiv.

[11]  William J. Godinez,et al.  Automatic analysis of dividing cells in live cell movies to detect mitotic delays and correlate phenotypes in time. , 2009, Genome research.

[12]  Juho Kannala,et al.  Joint cell segmentation and tracking using cell proposals , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[13]  Karl Rohr,et al.  Comparison of segmentation methods for tissue microscopy images of glioblastoma cells , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[14]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Tammy Riklin-Raviv,et al.  Microscopy cell segmentation via adversarial neural networks , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[16]  Lorenzo Rosasco,et al.  Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review , 2016, International Journal of Automation and Computing.

[17]  Christopher Joseph Pal,et al.  Delving Deeper into Convolutional Networks for Learning Video Representations , 2015, ICLR.

[18]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[19]  Marios Savvides,et al.  Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation , 2017, IEEE Transactions on Image Processing.

[20]  Karl Rohr,et al.  Adversarial domain adaptation to improve automatic breast cancer grading in lymph nodes , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[21]  Tomaso A. Poggio,et al.  Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex , 2016, ArXiv.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Karl Rohr,et al.  Segmentation of cell nuclei using intensity-based model fitting and sequential convex programming , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[27]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[28]  Carolina Wählby,et al.  Automated Training of Deep Convolutional Neural Networks for Cell Segmentation , 2017, Scientific Reports.

[29]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..

[30]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[31]  J A Sethian,et al.  A fast marching level set method for monotonically advancing fronts. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[32]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[33]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Joakim Jaldén,et al.  A batch algorithm using iterative application of the Viterbi algorithm to track cells and construct cell lineages , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[35]  Karl Rohr,et al.  Multi-channel Deep Transfer Learning for Nuclei Segmentation in Glioblastoma Cell Tissue Images , 2018, Bildverarbeitung für die Medizin.

[36]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Ana Maria Mendonça,et al.  Gradient convergence filters and a phase congruency approach for in vivo cell nuclei detection , 2012, Machine Vision and Applications.

[38]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[39]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Max Tegmark,et al.  Why Does Deep and Cheap Learning Work So Well? , 2016, Journal of Statistical Physics.

[42]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[43]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[44]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[45]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[46]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[48]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[49]  Karl Rohr,et al.  Deep residual Hough voting for mitotic cell detection in histopathology images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).