A probabilistic approach to joint cell tracking and segmentation in high‐throughput microscopy videos☆

HighlightsWe propose an unsupervised, automatic tracking and segmentation framework for high‐throughput microscopy image sequences.Cell segmentation and tracking are tied together via Bayesian inference of dynamic models.The Kalman inference problem is exploited to estimate the time‐wise cell shape uncertainty in addition to cell trajectory. The inferred cell properties are integrated with the observed image, using a fast marching algorithm, to obtain the image likelihood for cell segmentation and association.We present highly accurate results, surpassing the state of the art, for a variety of microscopy data sets with high dynamics, including long sequences (hundreds of frames). Graphical abstract Figure. No caption available. ABSTRACT We present a novel computational framework for the analysis of high‐throughput microscopy videos of living cells. The proposed framework is generally useful and can be applied to different datasets acquired in a variety of laboratory settings. This is accomplished by tying together two fundamental aspects of cell lineage construction, namely cell segmentation and tracking, via a Bayesian inference of dynamic models. In contrast to most existing approaches, which aim to be general, no assumption of cell shape is made. Spatial, temporal, and cross‐sectional variation of the analysed data are accommodated by two key contributions. First, time series analysis is exploited to estimate the temporal cell shape uncertainty in addition to cell trajectory. Second, a fast marching (FM) algorithm is used to integrate the inferred cell properties with the observed image measurements in order to obtain image likelihood for cell segmentation, and association. The proposed approach has been tested on eight different time‐lapse microscopy data sets, some of which are high‐throughput, demonstrating promising results for the detection, segmentation and association of planar cells. Our results surpass the state of the art for the Fluo‐C2DL‐MSC data set of the Cell Tracking Challenge (Maška et al., 2014).

[1]  Takeo Kanade,et al.  Cell image analysis: Algorithms, system and applications , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[2]  Koenraad Van Leemput,et al.  Joint Segmentation of Image Ensembles via Latent Atlases , 2009, MICCAI.

[3]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[4]  Fred A. Hamprecht,et al.  Conservation Tracking , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Takeo Kanade,et al.  Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features , 2013, Medical Image Anal..

[6]  Olaf Ronneberger,et al.  Cell segmentation and tracking in phase contrast images using graph cut with asymmetric boundary costs , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[7]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[8]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[9]  R Chakravorty,et al.  Automated and semi‐automated cell tracking: addressing portability challenges , 2011, Journal of microscopy.

[10]  K. Jaqaman,et al.  Robust single particle tracking in live cell time-lapse sequences , 2008, Nature Methods.

[11]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[13]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[14]  Pascal Fua,et al.  Globally Optimal Cell Tracking using Integer Programming , 2016 .

[15]  Christophe Zimmer,et al.  Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces , 2005, IEEE Transactions on Image Processing.

[16]  Ullrich Köthe,et al.  Graphical model for joint segmentation and tracking of multiple dividing cells , 2015, Bioinform..

[17]  Takeo Kanade,et al.  Reliable cell tracking by global data association , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[18]  Paul W. Fieguth,et al.  Extended-Hungarian-JPDA: Exact Single-Frame Stem Cell Tracking , 2007, IEEE Transactions on Biomedical Engineering.

[19]  Ronald Chung,et al.  The Multiplicative Path Toward Prior-Shape Guided Active Contour for Object Detection , 2007, ISVC.

[20]  Joakim Jalden,et al.  Global Linking of Cell Tracks Using the Viterbi Algorithm , 2015, IEEE Transactions on Medical Imaging.

[21]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..

[22]  Takeo Kanade,et al.  Cell population tracking and lineage construction with spatiotemporal context , 2008, Medical Image Anal..

[23]  Jean-Christophe Olivo-Marin,et al.  3-D Active Meshes: Fast Discrete Deformable Models for Cell Tracking in 3-D Time-Lapse Microscopy , 2011, IEEE Transactions on Image Processing.

[24]  Wiro J. Niessen,et al.  Advanced Level-Set-Based Cell Tracking in Time-Lapse Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[25]  Jens Rittscher,et al.  Spatio-temporal cell segmentation and tracking for automated screening , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  Serge Beucher,et al.  The Morphological Approach to Segmentation: The Watershed Transformation , 2018, Mathematical Morphology in Image Processing.

[28]  William F. Eddy,et al.  A novel algorithm for optimal image thresholding of biological data , 2010, Journal of Neuroscience Methods.

[29]  M. Maška,et al.  Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs , 2015, PloS one.

[30]  Karl Rohr,et al.  Direct combination of multi-scale detection and multi-frame association for tracking of virus particles in microscopy image data , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[31]  Brendan J. Frey,et al.  Classifying and segmenting microscopy images with deep multiple instance learning , 2015, Bioinform..

[32]  Pascal Fua,et al.  Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences , 2017, IEEE Transactions on Medical Imaging.

[33]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[34]  W. Eric L. Grimson,et al.  Logarithm Odds Maps for Shape Representation , 2006, MICCAI.

[35]  Xiaobo Zhou,et al.  Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[36]  Karl Rohr,et al.  Efficient globally optimal segmentation of cells in fluorescence microscopy images using level sets and convex energy functionals , 2012, Medical Image Anal..

[37]  Johannes E. Schindelin,et al.  The ImageJ ecosystem: An open platform for biomedical image analysis , 2015, Molecular reproduction and development.

[38]  Juho Kannala,et al.  Joint cell segmentation and tracking using cell proposals , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[39]  C Wählby,et al.  Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections , 2004, Journal of microscopy.

[40]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[41]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[42]  Anne E. Carpenter,et al.  Analysis of High-throughput Microscopy Videos: Catching Up with Cell Dynamics , 2015, MICCAI.

[43]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[44]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[45]  Aly A. Farag,et al.  MultiStencils Fast Marching Methods: A Highly Accurate Solution to the Eikonal Equation on Cartesian Domains , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Jens Rittscher,et al.  Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis , 2011, Medical Image Anal..

[47]  D. Rapoport,et al.  A Novel Validation Algorithm Allows for Automated Cell Tracking and the Extraction of Biologically Meaningful Parameters , 2011, PloS one.

[48]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[49]  Philipp J. Keller,et al.  Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data , 2014, Nature Methods.

[50]  R. Milo,et al.  Dynamic Proteomics of Individual Cancer Cells in Response to a Drug , 2008, Science.

[51]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[52]  Koenraad Van Leemput,et al.  Segmentation of image ensembles via latent atlases , 2010, Medical Image Anal..

[53]  Jean-Christophe Olivo-Marin,et al.  Improving active contours for segmentation and tracking of motile cells in videomicroscopy , 2002, Object recognition supported by user interaction for service robots.

[54]  Anne E. Carpenter,et al.  Symmetry-based mitosis detection in time-lapse microscopy , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).