Tracking Cells and Their Lineages Via Labeled Random Finite Sets

Determining the trajectories of cells and their lineages or ancestries in live-cell experiments are fundamental to the understanding of how cells behave and divide. This paper proposes novel online algorithms for jointly tracking and resolving lineages of an unknown and time-varying number of cells from time-lapse video data. Our approach involves modeling the cell ensemble as a labeled random finite set with labels representing cell identities and lineages. A spawning model is developed to take into account cell lineages and changes in cell appearance prior to division. We then derive analytic filters to propagate multi-object distributions that contain information on the current cell ensemble including their lineages. We also develop numerical implementations of the resulting multi-object filters. Experiments using simulation, synthetic cell migration video, and real time-lapse sequence, are presented to demonstrate the capability of the solutions.

[1]  David Suter,et al.  Visual tracking of numerous targets via multi-Bernoulli filtering of image data , 2012, Pattern Recognit..

[2]  Ba-Ngu Vo,et al.  CPHD Filtering With Unknown Clutter Rate and Detection Profile , 2011, IEEE Transactions on Signal Processing.

[3]  Ba-Ngu Vo,et al.  A Solution for Large-Scale Multi-Object Tracking , 2018, IEEE Transactions on Signal Processing.

[4]  Pascal Fua,et al.  Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences , 2017, IEEE Transactions on Medical Imaging.

[5]  Peter J. Park,et al.  Somatic mutation in single human neurons tracks developmental and transcriptional history , 2015, Science.

[6]  Daniel E. Clark,et al.  Marker-Less Stage Drift Correction in Super-Resolution Microscopy Using the Single-Cluster PHD Filter , 2016, IEEE Journal of Selected Topics in Signal Processing.

[7]  Yaakov Bar-Shalom,et al.  Sonar tracking of multiple targets using joint probabilistic data association , 1983 .

[8]  Du Yong Kim,et al.  On-line Tracking of Cells and Their Lineage from Time Lapse Video Data , 2018, 2018 International Conference on Control, Automation and Information Sciences (ICCAIS).

[9]  Ba-Ngu Vo,et al.  Bayesian Multi-Target Tracking With Merged Measurements Using Labelled Random Finite Sets , 2015, IEEE Transactions on Signal Processing.

[10]  Lars Relund Nielsen,et al.  An algorithm for ranking assignments using reoptimization , 2008, Comput. Oper. Res..

[11]  Ba-Ngu Vo,et al.  A generalized labeled multi-Bernoulli tracker for time lapse cell migration , 2017, 2017 International Conference on Control, Automation and Information Sciences (ICCAIS).

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[14]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[15]  Ba-Ngu Vo,et al.  Labeled Random Finite Sets and the Bayes Multi-Target Tracking Filter , 2013, IEEE Transactions on Signal Processing.

[16]  Ba-Ngu Vo,et al.  Multiple Object Tracking in Unknown Backgrounds With Labeled Random Finite Sets , 2017, IEEE Transactions on Signal Processing.

[17]  Ba Tuong Vo,et al.  A Multi-Scan Labeled Random Finite Set Model for Multi-Object State Estimation , 2018, IEEE Transactions on Signal Processing.

[18]  Ba-Ngu Vo,et al.  Labeled Random Finite Sets and Multi-Object Conjugate Priors , 2013, IEEE Transactions on Signal Processing.

[19]  Ba-Ngu Vo,et al.  Generalized Labeled Multi-Bernoulli Approximation of Multi-Object Densities , 2014, IEEE Transactions on Signal Processing.

[20]  P. Koumoutsakos,et al.  Feature point tracking and trajectory analysis for video imaging in cell biology. , 2005, Journal of structural biology.

[21]  Scott T. Acton,et al.  Level set analysis for leukocyte detection and tracking , 2004, IEEE Transactions on Image Processing.

[22]  Ba-Ngu Vo,et al.  An Efficient Implementation of the Generalized Labeled Multi-Bernoulli Filter , 2016, IEEE Transactions on Signal Processing.

[23]  Danny Ziyi Chen,et al.  A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms , 2015, IEEE Transactions on Medical Imaging.

[24]  Takeo Kanade,et al.  Cell population tracking and lineage construction with spatiotemporal context , 2008, Medical Image Anal..

[25]  Isabelle Bloch,et al.  Multiple Hypothesis Tracking for Cluttered Biological Image Sequences , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Ronald P. S. Mahler,et al.  Advances in Statistical Multisource-Multitarget Information Fusion , 2014 .

[27]  Benlian Xu,et al.  An Ant Colony Inspired Multi-Bernoulli Filter for Cell Tracking in Time-Lapse Microscopy Sequences , 2020, IEEE Journal of Biomedical and Health Informatics.

[28]  Takeshi Ishihara,et al.  SPF-CellTracker: Tracking Multiple Cells with Strongly-Correlated Moves Using a Spatial Particle Filter , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Klaus C. J. Dietmayer,et al.  The Labeled Multi-Bernoulli Filter , 2014, IEEE Transactions on Signal Processing.

[31]  Ba-Ngu Vo,et al.  A Generalized Labeled Multi-Bernoulli Filter With Object Spawning , 2017, IEEE Transactions on Signal Processing.

[32]  M. Omair Ahmad,et al.  Tracking Biological Cells in Time-Lapse Microscopy: An Adaptive Technique Combining Motion and Topological Features , 2011, IEEE Transactions on Biomedical Engineering.

[33]  Andrew Zisserman,et al.  Discriminative Semi-Markov Models for automated mitotic phase labelling , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[34]  Wiro J. Niessen,et al.  Particle Filtering for Multiple Object Tracking in Dynamic Fluorescence Microscopy Images: Application to Microtubule Growth Analysis , 2008, IEEE Transactions on Medical Imaging.

[35]  K. G. Murty An Algorithm for Ranking All the Assignment in Order of Increasing Cost , 1968 .

[36]  E. Meijering,et al.  Tracking in molecular bioimaging , 2006, IEEE Signal Processing Magazine.

[37]  Jean-Christophe Olivo-Marin,et al.  Extraction of spots in biological images using multiscale products , 2002, Pattern Recognit..

[38]  Ba-Ngu Vo,et al.  Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter , 2007, IEEE Transactions on Signal Processing.

[39]  Klas E. G. Magnusson,et al.  Segmentation and tracking of cells and particles in time-lapse microscopy , 2016 .

[40]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[41]  J.-Y. Lee,et al.  Deep Learning Particle Detection for Probabilistic Tracking in Fluorescence Microscopy Images , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[42]  M. Maška,et al.  Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs , 2015, PloS one.

[43]  Richard I. Hartley,et al.  A new approach for spot detection in total internal reflection fluorescence microscopy , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[44]  Gustavo Carneiro,et al.  An Improved Joint Optimization of Multiple Level Set Functions for the Segmentation of Overlapping Cervical Cells , 2015, IEEE Transactions on Image Processing.

[45]  Takeo Kanade,et al.  A Semi-Markov Model for Mitosis Segmentation in Time-Lapse Phase Contrast Microscopy Image Sequences of Stem Cell Populations , 2012, IEEE Transactions on Medical Imaging.

[46]  Philippe Van Ham,et al.  Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes , 2005, IEEE Transactions on Medical Imaging.

[47]  Joakim Jalden,et al.  Global Linking of Cell Tracks Using the Viterbi Algorithm , 2015, IEEE Transactions on Medical Imaging.

[48]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[49]  Fred A. Hamprecht,et al.  Conservation Tracking , 2013, 2013 IEEE International Conference on Computer Vision.

[50]  Ian D. Reid,et al.  Joint Probabilistic Data Association Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[51]  Dipti Prasad Mukherjee,et al.  Mitosis Detection for Invasive Breast Cancer Grading in Histopathological Images , 2015, IEEE Transactions on Image Processing.

[52]  Paul F. Whelan,et al.  A Novel Framework for Cellular Tracking and Mitosis Detection in Dense Phase Contrast Microscopy Images , 2013, IEEE Journal of Biomedical and Health Informatics.

[53]  Takeo Kanade,et al.  Automated Mitosis Detection of Stem Cell Populations in Phase-Contrast Microscopy Images , 2011, IEEE Transactions on Medical Imaging.

[54]  Wiro J. Niessen,et al.  Advanced Level-Set-Based Cell Tracking in Time-Lapse Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[55]  Du Yong Kim,et al.  GLMB Tracker with Partial Smoothing , 2019, Sensors.

[56]  Pamela C. Cosman,et al.  Automated cell junction tracking with modified active contours guided by SIFT flow , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[57]  Cong-Thanh Do,et al.  Multiple marine ships tracking from multistatic Doppler data with unknown clutter rate , 2019, 2019 International Conference on Control, Automation and Information Sciences (ICCAIS).

[58]  Pekka Ruusuvuori,et al.  Computational Framework for Simulating Fluorescence Microscope Images With Cell Populations , 2007, IEEE Transactions on Medical Imaging.

[59]  Scott E. Fraser,et al.  Imaging in Systems Biology , 2007, Cell.

[60]  Ba-Ngu Vo,et al.  Multi-Target Tracking With Time-Varying Clutter Rate and Detection Profile: Application to Time-Lapse Cell Microscopy Sequences , 2015, IEEE Transactions on Medical Imaging.

[61]  M.L. Miller,et al.  Optimizing Murty's ranked assignment method , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[62]  Jian Shi,et al.  A Joint Tracking Approach via Ant Colony Evolution for Quantitative Cell Cycle Analysis. , 2020, IEEE journal of biomedical and health informatics.

[63]  Amit K. Roy-Chowdhury,et al.  Context aware spatio-temporal cell tracking in densely packed multilayer tissues , 2015, Medical Image Anal..

[64]  A. Doucet,et al.  Sequential Monte Carlo methods for multitarget filtering with random finite sets , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[65]  Fatima Boukari,et al.  Automated Cell Tracking Using Motion Prediction-Based Matching and Event Handling , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[66]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Mei Chen,et al.  Mitosis Detection in Phase Contrast Microscopy Image Sequences of Stem Cell Populations: A Critical Review , 2017, IEEE Transactions on Big Data.