Joint Multi-frame Detection and Segmentation for Multi-cell Tracking

Tracking living cells in video sequence is difficult, because of cell morphology and high similarities between cells. Tracking-by-detection methods are widely used in multi-cell tracking. We perform multi-cell tracking based on the cell centroid detection, and the performance of the detector has high impact on tracking performance. In this paper, UNet is utilized to extract inter-frame and intra-frame spatio-temporal information of cells. Detection performance of cells in mitotic phase is improved by multi-frame input. Good detection results facilitate multi-cell tracking. A mitosis detection algorithm is proposed to detect cell mitosis and the cell lineage is built up. Another UNet is utilized to acquire primary segmentation. Jointly using detection and primary segmentation, cells can be fine segmented in highly dense cell population. Experiments are conducted to evaluate the effectiveness of our method, and results show its state-of-the-art performance.

[1]  Tobias Senst,et al.  Extending IOU Based Multi-Object Tracking by Visual Information , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[2]  Tammy Riklin-Raviv,et al.  Microscopy Cell Segmentation Via Convolutional LSTM Networks , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[5]  Wenhan Luo,et al.  Multiple Object Tracking: A Review , 2014, ArXiv.

[6]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[7]  Thomas Neff,et al.  Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks , 2018, MICCAI.

[8]  Milan Sonka,et al.  Cell Segmentation, Tracking, and Mitosis Detection Using Temporal Context , 2005, MICCAI.

[9]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Wenhan Luo,et al.  Multiple object tracking: A literature review , 2014, Artif. Intell..

[12]  Zhang Yi,et al.  Cell tracking using deep neural networks with multi-task learning , 2017, Image Vis. Comput..

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[15]  R. Khudeev,et al.  A new flood-fill algorithm for closed contour , 2005, 2005 Siberian Conference on Control and Communications.

[16]  Ling Xu,et al.  A generalized data association approach for cell tracking in high-density population , 2015, 2015 International Conference on Control, Automation and Information Sciences (ICCAIS).

[17]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[18]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[20]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[21]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[22]  Tammy Riklin-Raviv,et al.  Microscopy cell segmentation via adversarial neural networks , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[23]  Christopher Joseph Pal,et al.  Delving Deeper into Convolutional Networks for Learning Video Representations , 2015, ICLR.

[24]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..