Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences

We propose a novel approach to automatically tracking elliptical cell populations in time-lapse image sequences. Given an initial segmentation, we account for partial occlusions and overlaps by generating an over-complete set of competing detection hypotheses. To this end, we fit ellipses to portions of the initial regions and build a hierarchy of ellipses, which are then treated as cell candidates. We then select temporally consistent ones by solving to optimality an integer program with only one type of flow variables. This eliminates the need for heuristics to handle missed detections due to partial occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging sequences consisting of clumped cells and show that it outperforms state-of-the-art techniques.

[1]  J. Friedman Stochastic gradient boosting , 2002 .

[2]  James C. Gee,et al.  Biomedical Image Registration , 2003, Lecture Notes in Computer Science.

[3]  James J. Little,et al.  A Linear Programming Approach for Multiple Object Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Robert T. Collins,et al.  Multi-target Data Association by Tracklets with Unsupervised Parameter Estimation , 2008, BMVC.

[5]  Takeo Kanade,et al.  Cell population tracking and lineage construction with spatiotemporal context , 2008, Medical Image Anal..

[6]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[7]  Jens Rittscher,et al.  Coupled Minimum-Cost Flow Cell Tracking , 2009, IPMI.

[8]  Jing Zhang,et al.  Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jens Rittscher,et al.  Spatio-temporal cell cycle phase analysis using level sets and fast marching methods , 2009, Medical Image Anal..

[10]  Konrad Schindler,et al.  Globally Optimal Multi-target Tracking on a Hexagonal Lattice , 2010, ECCV.

[11]  Wiro J. Niessen,et al.  Advanced Level-Set-Based Cell Tracking in Time-Lapse Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[12]  Xiaobo Zhou,et al.  Multiple Nuclei Tracking Using Integer Programming for Quantitative Cancer Cell Cycle Analysis , 2010, IEEE Transactions on Medical Imaging.

[13]  Bernd Fischer,et al.  Biomedical Image Registration, 4th International Workshop, WBIR 2010, Lübeck, Germany, July 11-13, 2010. Proceedings , 2010, Workshop on Biomedical Image Registration.

[14]  Jean-Christophe Olivo-Marin,et al.  3-D Active Meshes: Fast Discrete Deformable Models for Cell Tracking in 3-D Time-Lapse Microscopy , 2011, IEEE Transactions on Image Processing.

[15]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[16]  Ullrich Köthe,et al.  Ilastik: Interactive learning and segmentation toolkit , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[17]  Jens Rittscher,et al.  Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis , 2011, Medical Image Anal..

[18]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[19]  Luc Van Gool,et al.  Hough Forests for Object Detection, Tracking, and Action Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Eric L. Miller,et al.  Segmentation fusion for connectomics , 2011, 2011 International Conference on Computer Vision.

[21]  Michael Unser,et al.  Fast parametric snakes for 3D microscopy , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[22]  Adrien Bartoli,et al.  Tracking by Detection for Interactive Image Augmentation in Laparoscopy , 2012, WBIR.

[23]  Konrad Schindler,et al.  Discrete-continuous optimization for multi-target tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Ramakant Nevatia,et al.  An online learned CRF model for multi-target tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Joakim Jaldén,et al.  A batch algorithm using iterative application of the Viterbi algorithm to track cells and construct cell lineages , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[26]  Erik Meijering,et al.  Methods for cell and particle tracking. , 2012, Methods in enzymology.

[27]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[29]  Matthew Cook,et al.  Efficient automatic 3D-reconstruction of branching neurons from EM data , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Ullrich Köthe,et al.  A Discrete Chain Graph Model for 3d+t Cell Tracking with High Misdetection Robustness , 2012, ECCV.

[31]  Fred A. Hamprecht,et al.  Conservation Tracking , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Carlos Ortiz-de-Solorzano,et al.  Segmentation and Shape Tracking of Whole Fluorescent Cells Based on the Chan–Vese Model , 2013, IEEE Transactions on Medical Imaging.

[33]  Hiok Chai Quek,et al.  ElliFit: An unconstrained, non-iterative, least squares based geometric Ellipse Fitting method , 2013, Pattern Recognit..

[34]  Nassir Navab,et al.  Interventional Tool Tracking Using Discrete Optimization , 2013, IEEE Transactions on Medical Imaging.

[35]  Stephen T. C. Wong,et al.  Chapter 17: Bioimage Informatics for Systems Pharmacology , 2013, PLoS Comput. Biol..

[36]  Bernt Schiele,et al.  Monocular Visual Scene Understanding: Understanding Multi-Object Traffic Scenes , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Eugene W. Myers,et al.  Optimal Joint Segmentation and Tracking of Escherichia Coli in the Mother Machine , 2014, BAMBI.

[38]  Nathalie Harder,et al.  A benchmark for comparison of cell tracking algorithms , 2014, Bioinform..

[39]  Shai Avidan,et al.  Extended Lucas-Kanade Tracking , 2014, ECCV.

[40]  William J. Godinez,et al.  Objective comparison of particle tracking methods , 2014, Nature Methods.

[41]  David Zhang,et al.  Fast Visual Tracking via Dense Spatio-temporal Context Learning , 2014, ECCV.

[42]  Philipp J. Keller,et al.  Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data , 2014, Nature Methods.

[43]  Robert T. Collins,et al.  Hybrid Stochastic / Deterministic Optimization for Tracking Sports Players and Pedestrians , 2014, ECCV.

[44]  Stan Sclaroff,et al.  MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization , 2014, ECCV.

[45]  Pascal Fua,et al.  Tracking Interacting Objects Optimally Using Integer Programming , 2014, ECCV.

[46]  Pascal Fua,et al.  Take your eyes off the ball: Improving ball-tracking by focusing on team play , 2014, Comput. Vis. Image Underst..

[47]  Harjeet Singh,et al.  Automated profiling of individual cell-cell interactions from high-throughput time-lapse imaging microscopy in nanowell grids (TIMING) , 2015, Bioinform..

[48]  M. Maška,et al.  Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs , 2015, PloS one.

[49]  Ullrich Köthe,et al.  Graphical model for joint segmentation and tracking of multiple dividing cells , 2015, Bioinform..

[50]  O. Troyanskaya,et al.  Video Bioinformatics , 2015, Computational Biology.

[51]  Florian Jug,et al.  Tracking by Assignment Facilitates Data Curation , 2015, MICCAI 2015.

[52]  Badrinath Roysam,et al.  Integrated 5-D Cell Tracking and Linked Analytics in the FARSIGHT Open Source Toolkit , 2015 .

[53]  Mathieu Salzmann,et al.  A Linear Chain Markov Model for Detection and Localization of Cells in Early Stage Embryo Development , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[54]  Joakim Jalden,et al.  Global Linking of Cell Tracks Using the Viterbi Algorithm , 2015, IEEE Transactions on Medical Imaging.

[55]  Pascal Fua,et al.  What Players do with the Ball: A Physically Constrained Interaction Modeling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Pascal Fua,et al.  Globally Consistent Multi-People Tracking using Motion Patterns , 2016, ArXiv.

[57]  Pascal Fua,et al.  Tracking Interacting Objects Using Intertwined Flows , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.