Machine learning enhanced cell tracking

Quantifying cell biology in space and time requires computational methods to detect cells, measure their properties, and assemble these into meaningful trajectories. In this aspect, machine learning (ML) is having a transformational effect on bioimage analysis, now enabling robust cell detection in multidimensional image data. However, the task of cell tracking, or constructing accurate multi-generational lineages from imaging data, remains an open challenge. Most cell tracking algorithms are largely based on our prior knowledge of cell behaviors, and as such, are difficult to generalize to new and unseen cell types or datasets. Here, we propose that ML provides the framework to learn aspects of cell behavior using cell tracking as the task to be learned. We suggest that advances in representation learning, cell tracking datasets, metrics, and methods for constructing and evaluating tracking solutions can all form part of an end-to-end ML-enhanced pipeline. These developments will lead the way to new computational methods that can be used to understand complex, time-evolving biological systems.

[1]  Noor M. Al-Shakarji,et al.  The Cell Tracking Challenge: 10 years of objective benchmarking , 2023, Nature Methods.

[2]  M. E. Coston,et al.  Integrated intracellular organization and its variations in human iPS cells , 2023, Nature.

[3]  Matthew P. Neilson,et al.  Traject3d allows label-free identification of distinct co-occurring phenotypes within 3D culture by live imaging , 2022, Nature Communications.

[4]  Alan R. Lowe,et al.  Learning biophysical determinants of cell fate with deep neural networks , 2022, Nature Machine Intelligence.

[5]  Romain F. Laine,et al.  TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines , 2022, Nature Methods.

[6]  Katie Cockburn,et al.  Probing the rules of cell coordination in live tissues by interpretable machine learning based on graph neural networks , 2022, bioRxiv.

[7]  Loic A. Royer,et al.  DaXi—high-resolution, large imaging volume and multi-view single-objective light-sheet microscopy , 2022, Nature Methods.

[8]  Yinan Wan,et al.  Current approaches to fate mapping and lineage tracing using image data. , 2021, Development.

[9]  Anne E Carpenter,et al.  CellProfiler 4: improvements in speed, utility and usability , 2021, BMC Bioinformatics.

[10]  Daniel Coelho de Castro,et al.  Active label cleaning for improved dataset quality under resource constraints , 2021, Nature Communications.

[11]  Philipp J. Keller,et al.  Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations , 2021, bioRxiv.

[12]  Erik S. Welf,et al.  Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. , 2021, Cell systems.

[13]  M. Averof,et al.  Tracking cell lineages in 3D by incremental deep learning , 2021, bioRxiv.

[14]  Marius Pachitariu,et al.  Cellpose: a generalist algorithm for cellular segmentation , 2020, Nature Methods.

[15]  Guillaume Charras,et al.  Automated deep lineage tree analysis using a Bayesian single cell tracking approach , 2020, bioRxiv.

[16]  Shalin B. Mehta,et al.  DynaMorph: self-supervised learning of morphodynamic states of live cells , 2021 .

[17]  E. Paluch,et al.  Single-cell morphometrics reveals ancestral principles of notochord development , 2020, bioRxiv.

[18]  Samreen Anjum,et al.  CTMC: Cell Tracking with Mitosis Detection Dataset Challenge , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Erik S. Welf,et al.  A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics , 2020, bioRxiv.

[20]  Jiahe Li,et al.  Graph Networks for Multiple Object Tracking , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[21]  T. Höfer,et al.  Hidden long-range memories of growth and cycle speed correlate cell cycles in lineage trees , 2020, eLife.

[22]  G. Martius,et al.  Differentiation of Blackbox Combinatorial Solvers , 2019, ICLR.

[23]  Alan R. Lowe,et al.  Cell-scale biophysical determinants of cell competition in epithelia , 2019, bioRxiv.

[24]  Erik S. Welf,et al.  Robust and automated detection of subcellular morphological motifs in 3D microscopy images , 2019, Nature Methods.

[25]  William Graf,et al.  Deep learning for cellular image analysis , 2019, Nature Methods.

[26]  Casper F Winsnes,et al.  Deep learning is combined with massive-scale citizen science to improve large-scale image classification , 2018, Nature Biotechnology.

[27]  Eugene W. Myers,et al.  Cell Detection with Star-convex Polygons , 2018, MICCAI.

[28]  Anne E Carpenter,et al.  Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images , 2018, bioRxiv.

[29]  Léo Guignard,et al.  Multi-view light-sheet imaging and tracking with the MaMuT software reveals the cell lineage of a direct developing arthropod limb , 2018, eLife.

[30]  Alan R. Lowe,et al.  Local cellular neighborhood controls proliferation in cell competition , 2017, Molecular biology of the cell.

[31]  Nathalie Harder,et al.  An Objective Comparison of Cell Tracking Algorithms , 2017, Nature Methods.

[32]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[33]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Johannes Schindelin,et al.  TrackMate: An open and extensible platform for single-particle tracking. , 2017, Methods.

[35]  Euan A. Ashley,et al.  Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments , 2016, PLoS Comput. Biol..

[36]  M. Maška,et al.  Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs , 2015, PloS one.

[37]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Joakim Jalden,et al.  Global Linking of Cell Tracks Using the Viterbi Algorithm , 2015, IEEE Transactions on Medical Imaging.

[39]  Itamar Simon,et al.  Lineage correlations of single cell division time as a probe of cell-cycle dynamics , 2015, Nature.

[40]  Wesley R. Legant,et al.  Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution , 2014, Science.

[41]  Alexandra Bokinsky,et al.  Dual-view plane illumination microscopy for rapid and spatially isotropic imaging , 2014, Nature Protocols.

[42]  Philipp J. Keller,et al.  Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data , 2014, Nature Methods.

[43]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[44]  Takeo Kanade,et al.  Reliable cell tracking by global data association , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[45]  Bernd Fischer,et al.  CellCognition: time-resolved phenotype annotation in high-throughput live cell imaging , 2010, Nature Methods.

[46]  Christopher Dunsby,et al.  Optically sectioned imaging by oblique plane microscopy , 2008, European Conference on Biomedical Optics.

[47]  Takeo Kanade,et al.  Cell population tracking and lineage construction with spatiotemporal context , 2008, Medical Image Anal..

[48]  K. Jaqaman,et al.  Robust single particle tracking in live cell time-lapse sequences , 2008, Nature Methods.

[49]  John Isaac Murray,et al.  The lineaging of fluorescently-labeled Caenorhabditis elegans embryos with StarryNite and AceTree , 2006, Nature Protocols.

[50]  B. Roysam,et al.  Automated Cell Lineage Construction: A Rapid Method to Analyze Clonal Development Established with Murine Neural Progenitor Cells , 2006, Cell cycle.

[51]  D. Grier,et al.  Methods of Digital Video Microscopy for Colloidal Studies , 1996 .

[52]  A. Volgenant,et al.  A shortest augmenting path algorithm for dense and sparse linear assignment problems , 1987, Computing.

[53]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[54]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[55]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.