Deep-learning method for data association in particle tracking

MOTIVATION Biological studies of dynamic processes in living cells often require accurate particle tracking as a first step toward quantitative analysis. Although many particle tracking methods have been developed for this purpose, they are typically based on prior assumptions about the particle dynamics, and/or they involve careful tuning of various algorithm parameters by the user for each application. This may make existing methods difficult to apply by non-expert users and to a broader range of tracking problems. Recent advances in deep-learning techniques hold great promise in eliminating these disadvantages, as they can learn how to optimally track particles from example data. RESULTS Here, we present a deep-learning-based method for the data association stage of particle tracking. The proposed method uses convolutional neural networks and long short-term memory networks to extract relevant dynamics features and predict the motion of a particle and the cost of linking detected particles from one time point to the next. Comprehensive evaluations on datasets from the particle tracking challenge demonstrate the competitiveness of the proposed deep-learning method compared to the state of the art. Additional tests on real-time-lapse fluorescence microscopy images of various types of intracellular particles show the method performs comparably with human experts. AVAILABILITY AND IMPLEMENTATION The software code implementing the proposed method as well as a description of how to obtain the test data used in the presented experiments will be available for non-commercial purposes from https://github.com/yoyohoho0221/pt_linking. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.

[1]  Erik Meijering,et al.  Methods for cell and particle tracking. , 2012, Methods in enzymology.

[2]  Yi Yang,et al.  Multiple dense particle tracking in fluorescence microscopy images based on multidimensional assignment. , 2011, Journal of structural biology.

[3]  Chao Li,et al.  Developing Noise-Resistant Three-Dimensional Single Particle Tracking Using Deep Neural Networks. , 2018, Analytical chemistry.

[4]  Karl Rohr,et al.  Performance and sensitivity evaluation of 3D spot detection methods in confocal microscopy , 2015, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[5]  Ola Spjuth,et al.  Deep Learning in Image Cytometry: A Review , 2018, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[6]  Erik Meijering,et al.  EB1 and EB3 regulate microtubule minus end organization and Golgi morphology , 2017, The Journal of cell biology.

[7]  P. J. Huber Robust Estimation of a Location Parameter , 1964 .

[8]  Michael J Saxton,et al.  Single-particle tracking: connecting the dots , 2008, Nature Methods.

[9]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[10]  Jean-Baptiste Sibarita,et al.  High-density single-particle tracking: quantifying molecule organization and dynamics at the nanoscale , 2014, Histochemistry and Cell Biology.

[11]  Nicholas A Moringo,et al.  Single Particle Tracking: From Theory to Biophysical Applications. , 2017, Chemical reviews.

[12]  Akira Funahashi,et al.  Predicting the future direction of cell movement with convolutional neural networks , 2019, PloS one.

[13]  Anna Akhmanova,et al.  Control of endothelial cell polarity and sprouting angiogenesis by non-centrosomal microtubules , 2018, eLife.

[14]  Prabhat,et al.  The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking , 2017 .

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[17]  Zhang Yi,et al.  Cell tracking using deep neural networks with multi-task learning , 2017, Image Vis. Comput..

[18]  Erik H. W. Meijering,et al.  Quantitative comparison of multiframe data association techniques for particle tracking in time-lapse fluorescence microscopy , 2015, Medical Image Anal..

[19]  William J. Godinez,et al.  Objective comparison of particle tracking methods , 2014, Nature Methods.

[20]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[21]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[22]  Wiro J. Niessen,et al.  Quantitative Comparison of Spot Detection Methods in Fluorescence Microscopy , 2010, IEEE Transactions on Medical Imaging.

[23]  Brendan J. Frey,et al.  Classifying and segmenting microscopy images with deep multiple instance learning , 2015, Bioinform..

[24]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[25]  Erik Meijering,et al.  Automated Analysis of Intracellular Dynamic Processes. , 2017, Methods in molecular biology.

[26]  K. Jaqaman,et al.  Robust single particle tracking in live cell time-lapse sequences , 2008, Nature Methods.

[27]  Pekka Ruusuvuori,et al.  Open Access Research Article Evaluation of Methods for Detection of Fluorescence Labeled Subcellular Objects in Microscope Images , 2022 .

[28]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[29]  Euan A. Ashley,et al.  Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments , 2016, PLoS Comput. Biol..

[30]  M Gregory Forest,et al.  Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D , 2017, Proceedings of the National Academy of Sciences.