Label-free prediction of three-dimensional fluorescence images from transmitted light microscopy

Understanding cells as integrated systems is a challenge central to modern biology. While different microscopy approaches may be used to probe diverse aspects of biological organization, each method presents limitations which ultimately restrict a view into unified cellular organization. For example, while fluorescence microscopy can resolve subcellular structure in living cells, it is expensive, slow, and can damage cells. Here, we present a label-free method for predicting 3D fluorescence directly from transmitted light images and demonstrate that it can be used to generate multi-structure, integrated images. We then demonstrate that this same method can be used to predict immunofluorescence from electron micrograph inputs, extending the method to a wider range of bioimaging applications.

[1]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[2]  Samuel J. Yang,et al.  In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images , 2018, Cell.

[3]  R. Wollman,et al.  Genes Required for Mitotic Spindle Assembly in Drosophila S2 Cells , 2007, Science.

[4]  Stavroula Skylaki,et al.  Challenges in long-term imaging and quantification of single-cell dynamics , 2016, Nature Biotechnology.

[5]  Suraiya Rasheed,et al.  Characterization of a newly derived human sarcoma cell line (HT‐1080) , 1974, Cancer.

[6]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[7]  F. Graham,et al.  Characteristics of a human cell line transformed by DNA from human adenovirus type 5. , 1977, The Journal of general virology.

[8]  Johannes E. Schindelin,et al.  TrakEM2 Software for Neural Circuit Reconstruction , 2012, PloS one.

[9]  Loic A. Royer,et al.  Content-aware image restoration: pushing the limits of fluorescence microscopy , 2018, Nature Methods.

[10]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[11]  Joy Arakaki,et al.  Systematic gene tagging using CRISPR/Cas9 in human stem cells to illuminate cell organization , 2017, Molecular biology of the cell.

[12]  Wesley R. Legant,et al.  Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution , 2014, Science.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[15]  Mark Bates,et al.  Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging , 2011, Nature Methods.

[16]  Pekka Ruusuvuori,et al.  Bright Field Microscopy as an Alternative to Whole Cell Fluorescence in Automated Analysis of Macrophage Images , 2009, PloS one.

[17]  Kristina D Micheva,et al.  Mapping Synapses by Conjugate Light-Electron Array Tomography , 2015, The Journal of Neuroscience.

[18]  Christopher J. Peddie,et al.  3D correlative light and electron microscopy of cultured cells using serial blockface scanning electron microscopy , 2017, Journal of Cell Science.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[21]  V. Magidson,et al.  Circumventing photodamage in live-cell microscopy. , 2013, Methods in cell biology.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[24]  Harald F. Hess,et al.  Correlative Photoactivated Localization and Scanning Electron Microscopy , 2013, PloS one.