Extracting quantitative biological information from bright-field cell images using deep learning

Saga Helgadottir1,∗, Benjamin Midtvedt1,∗, Jesús Pineda1,∗, Alan Sabirsh, Caroline B. Adiels, Stefano Romeo, Daniel Midtvedt, and Giovanni Volpe ∗ These authors contributed equally. Department of Physics, University of Gothenburg, Sweden Advanced Drug Delivery, Pharmaceutical Sciences, R&D, AstraZeneca, Gothenburg, Sweden Department of Molecular and Clinical Medicine, Institute of Medicine, Sahlgrenska Academy, Wallenberg Laboratory, University of Gothenburg, Sweden Department of Cardiology, Sahlgrenska University Hospital, Sweden and Clinical Nutrition Unit, Department of Medical and Surgical Sciences, University Magna Graecia, Italy (Dated: December 25, 2020)

[1]  Tullio Pozzan,et al.  Chimeric green fluorescent protein as a tool for visualizing subcellular organelles in living cells , 1995, Current Biology.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  H. Robenek,et al.  Lipids partition caveolin‐1 from ER membranes into lipid droplets: updating the model of lipid droplet biogenesis , 2004, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[4]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[5]  Jussi Taipale,et al.  Identification of pathways regulating cell size and cell-cycle progression by RNAi , 2006, Nature.

[6]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2018, Nature Biomedical Engineering.

[7]  Shuiwang Ji,et al.  Global Pixel Transformers for Virtual Staining of Microscopy Images , 2019, IEEE Transactions on Medical Imaging.

[8]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[9]  Yibo Zhang,et al.  PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning , 2018, Light: Science & Applications.

[10]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[11]  Valery V Tuchin,et al.  Refractive index of adipose tissue and lipid droplet measured in wide spectral and temperature ranges. , 2018, Applied optics.

[12]  A. Sabirsh,et al.  Effect of complexing lipids on cellular uptake and expression of messenger RNA in human skin explants. , 2020, Journal of controlled release : official journal of the Controlled Release Society.

[13]  Anne E Carpenter,et al.  CellProfiler 3.0: Next-generation image processing for biology , 2018, PLoS biology.

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  Mark D Hannel,et al.  Machine-learning techniques for fast and accurate feature localization in holograms of colloidal particles. , 2018, Optics express.

[16]  Apostolos Delis,et al.  Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue , 2020, Light, science & applications.

[17]  Jie Tian,et al.  Deep Learning for Virtual Histological Staining of Bright-Field Microscopic Images of Unlabeled Carotid Artery Tissue , 2020, Molecular Imaging and Biology.

[18]  A. Ozcan,et al.  On the use of deep learning for computational imaging , 2019, Optica.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Simcha K. Mirsky,et al.  Holographic virtual staining of individual biological cells , 2020, Proceedings of the National Academy of Sciences.

[21]  M Gregory Forest,et al.  Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D , 2017, Proceedings of the National Academy of Sciences.

[22]  E. Tsochatzis,et al.  The multiple-hit pathogenesis of non-alcoholic fatty liver disease (NAFLD). , 2016, Metabolism: clinical and experimental.

[23]  Gang-Yu Liu,et al.  Cell tracing dyes significantly change single cell mechanics. , 2009, The journal of physical chemistry. B.

[24]  Gabriel Popescu,et al.  Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments , 2020, Nature Communications.

[25]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[26]  Guillaume Charras,et al.  A short history of blebbing , 2008, Journal of microscopy.

[27]  A. Roli.  Artificial Neural Networks , 2012, Lecture Notes in Computer Science.

[28]  Xin Li,et al.  Wavelet Feature Outdoor Fingerprint Localization Based on ResNet and Deep Convolution GAN , 2020, Symmetry.

[29]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Aydogan Ozcan,et al.  Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram , 2018, Light: Science & Applications.

[32]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[33]  Giovanni Volpe,et al.  Quantitative digital microscopy with deep learning , 2020, Applied Physics Reviews.

[34]  V. Schrauwen-Hinderling,et al.  Lipid accumulation in non-adipose tissue and lipotoxicity , 2008, Physiology & Behavior.

[35]  Gregory R. Johnson,et al.  Label-free prediction of three-dimensional fluorescence images from transmitted light microscopy , 2018, Nature Methods.

[36]  Martin Purschke,et al.  Phototoxicity of Hoechst 33342 in time-lapse fluorescence microscopy , 2010, Photochemical & photobiological sciences : Official journal of the European Photochemistry Association and the European Society for Photobiology.

[37]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[38]  Junying Yuan,et al.  Cell death assays for drug discovery , 2011, Nature Reviews Drug Discovery.

[39]  Frank Cichos,et al.  Machine learning for active matter , 2020, Nat. Mach. Intell..

[40]  Giovanni Volpe,et al.  Digital video microscopy enhanced by deep learning , 2018, Optica.

[41]  L. Tian,et al.  Single-cell cytometry via multiplexed fluorescence prediction by label-free reflectance microscopy , 2020, Science Advances.

[42]  P. Seale,et al.  Thermogenic activity of UCP1 in human white fat-derived beige adipocytes. , 2015, Molecular endocrinology.

[43]  Hisataka Kobayashi,et al.  Toxicity of Organic Fluorophores Used in Molecular Imaging: Literature Review , 2009, Molecular imaging.

[44]  J. Guck,et al.  Cell nuclei have lower refractive index and mass density than cytoplasm , 2016, Journal of biophotonics.

[45]  T. Zimmermann.  Spectral imaging and linear unmixing in light microscopy. , 2005, Advances in biochemical engineering/biotechnology.