An accurate and universal approach for short-exposure-time microscopy image enhancement

Fluorescence microscopy imaging has become an essential technique in the biology and biomedical science which can provide comprehensive visualization of many biological processes, and the exposure time is one of the most critical parameters for fluorescence microscopy imaging. Short-exposure-time (SET) imaging overcomes the limitations of photo-bleaching and photo-toxicity, allowing comprehensive visualization of the biological processes. Unfortunately, SET images deteriorate the signal to noise ratio and the image quality when compared with the long-exposure-time (LET) images. Therefore, we introduce a data-driven microscopy image enhancement network (MIEN) to improve the quality of SET microscopy images without requiring any manual intervention, facilitating the production of high-resolution and high contrast images. The universal property and accuracy of the proposed network are validated on 38,500 real fluorescence microscopy images, which contain different object contents and are collected from various exposure time ratios and fluorescence microscopes platforms. Experimental results demonstrate that the proposed MIEN model is effective to enhance the quality of SET fluorescence microscopy images, and can be used to observe delicate changes in cells, tissues and organs with low photo-bleaching and photo-toxicity.

[1]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[2]  Claire M Brown,et al.  Fluorescence microscopy - avoiding the pitfalls , 2007, Journal of Cell Science.

[3]  Eric L. Miller,et al.  Joint volumetric extraction and enhancement of vasculature from low-SNR 3-D fluorescence microscopy images , 2017, Pattern Recognit..

[4]  G. J. Brakenhoff,et al.  Three-dimensional chromatin distribution in neuroblastoma nuclei shown by confocal scanning laser microscopy , 1985, Nature.

[5]  Ernst H K Stelzer,et al.  Light-sheet fluorescence microscopy for quantitative biology , 2014, Nature Methods.

[6]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[7]  D. Axelrod.  Total Internal Reflection Fluorescence Microscopy in Cell Biology , 2001, Traffic.

[8]  Hanchuan Peng,et al.  V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets , 2010, Nature Biotechnology.

[9]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  E. Manders,et al.  Controlled light-exposure microscopy reduces photobleaching and phototoxicity in fluorescence live-cell imaging , 2007, Nature Biotechnology.

[11]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[12]  Luc Duong,et al.  CycleGAN for style transfer in X-ray angiography , 2019, International Journal of Computer Assisted Radiology and Surgery.

[13]  Jun Wang,et al.  Attention-Aware Multi-Stroke Style Transfer , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jennifer C. Waters,et al.  Accuracy and precision in quantitative fluorescence microscopy , 2009, The Journal of cell biology.

[15]  K. Jaqaman,et al.  Robust single particle tracking in live cell time-lapse sequences , 2008, Nature Methods.

[16]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Jerry L. Prince,et al.  Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data size , 2018, SASHIMI@MICCAI.

[18]  Benjamin Berkels,et al.  Joint denoising and distortion correction of atomic scale scanning transmission electron microscopy images , 2016, ArXiv.

[19]  Laura Waller,et al.  Standardizing the resolution claims for coherent microscopy , 2016, Nature Photonics.

[20]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  L. Popescu,et al.  Extracellular vesicles release by cardiac telocytes: electron microscopy and electron tomography , 2014, Journal of cellular and molecular medicine.

[22]  Lihong V. Wang,et al.  Functional photoacoustic microscopy for high-resolution and noninvasive in vivo imaging , 2006, Nature Biotechnology.

[23]  Nathalie Harder,et al.  Automated Analysis of the Mitotic Phases of Human Cells in 3D Fluorescence Microscopy Image Sequences , 2006, MICCAI.

[24]  Torsten Wittmann,et al.  Fluorescence live cell imaging. , 2014, Methods in cell biology.

[25]  Philipp J. Keller,et al.  Fast, high-contrast imaging of animal development with scanned light sheet–based structured-illumination microscopy , 2010, Nature Methods.

[26]  Nilanjan Dey,et al.  Light microscopy image de-noising using optimized LPA-ICI filter , 2018, Neural Computing and Applications.

[27]  Eric T Ahrens,et al.  In vivo imaging platform for tracking immunotherapeutic cells , 2005, Nature Biotechnology.

[28]  Eugene W. Myers,et al.  Unsupervised segmentation of noisy electron microscopy images using salient watersheds and region merging , 2013, BMC Bioinformatics.

[29]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[30]  S. Pennycook,et al.  Atom-by-atom structural and chemical analysis by annular dark-field electron microscopy , 2010, Nature.

[31]  Amit Singer,et al.  Denoising and Covariance Estimation of Single Particle Cryo-EM Images , 2016, Journal of structural biology.

[32]  M. Gustafsson.  Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Claire M. Brown,et al.  Less is More: Longer Exposure Times with Low Light Intensity is Less Photo-Toxic , 2017, Microscopy Today.

[34]  Ioannis A. Kakadiaris,et al.  Denoising for 3-D Photon-Limited Imaging Data Using Nonseparable Filterbanks , 2008, IEEE Transactions on Image Processing.

[35]  P. Santi,et al.  Light Sheet Fluorescence Microscopy , 2011, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.