Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images

Super-resolution structured illumination microscopy (SR-SIM) provides an up to two-fold enhanced spatial resolution of fluorescently labeled samples. The reconstruction of high quality SR-SIM images critically depends on patterned illumination with high modulation contrast. Noisy raw image data, e.g. as a result of low excitation power or low exposure times, result in reconstruction artifacts. Here, we demonstrate deep-learning based SR-SIM image denoising that results in high quality reconstructed images. A residual encoding-decoding convolution neural network (RED-Net) was used to successfully denoise computationally reconstructed noisy SR-SIM images. We also demonstrate the entirely deep-learning based denoising and reconstruction of raw SIM images into high-resolution SR-SIM images. Both image reconstruction methods prove to be very robust against image reconstruction artifacts and generalize very well over various noise levels. The combination of computational reconstruction and subsequent denoising via RED-Net shows very robust performance during inference after training even if the microscope settings change.

[1]  Maria Smedh,et al.  Successful optimization of reconstruction parameters in structured illumination microscopy – a practical guide , 2018 .

[2]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Emilio Soria Olivas,et al.  Handbook of Research on Machine Learning Applications and Trends : Algorithms , Methods , and Techniques , 2009 .

[6]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[7]  Suliana Manley,et al.  Quantitative evaluation of software packages for single-molecule localization microscopy , 2015, Nature Methods.

[8]  Pietro Lio,et al.  ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images , 2020, ArXiv.

[9]  T. Huser,et al.  DMD-based super-resolution structured illumination microscopy visualizes live cell dynamics at high speed and low cost , 2019, bioRxiv.

[10]  Bryant B. Chhun,et al.  Super-Resolution Video Microscopy of Live Cells by Structured Illumination , 2009, Nature Methods.

[11]  Liangyi Chen,et al.  A protocol for structured illumination microscopy with minimal reconstruction artifacts , 2019, Biophysics Reports.

[12]  Fenqiang Zhao,et al.  Deep learning enables structured illumination microscopy with low light levels and enhanced speed , 2019, Nature Communications.

[13]  O. Mandula,et al.  Structured illumination microscopy of a living cell , 2009, 2011 International Quantum Electronics Conference (IQEC) and Conference on Lasers and Electro-Optics (CLEO) Pacific Rim incorporating the Australasian Conference on Optics, Lasers and Spectroscopy and the Australian Conference on Optical Fibre Technology.

[14]  Tibor Novák,et al.  TestSTORM: Versatile simulator software for multimodal super-resolution localization fluorescence microscopy , 2017, Scientific Reports.

[15]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[16]  C. Felser,et al.  Negative magnetoresistance without well-defined chirality in the Weyl semimetal TaP , 2015, Nature Communications.

[17]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[18]  P. Xi,et al.  Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy , 2018, Nature Biotechnology.

[19]  M. Davidson,et al.  Noninvasive Imaging beyond the Diffraction Limit of 3D Dynamics in Thickly Fluorescent Specimens , 2012, Cell.

[20]  M. Kunitski,et al.  Double-slit photoelectron interference in strong-field ionization of the neon dimer , 2018, Nature Communications.

[21]  Karel Fliegel,et al.  SIMToolbox: a MATLAB toolbox for structured illumination fluorescence microscopy , 2015, Bioinform..

[22]  Thomas Huser,et al.  Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction , 2019, Nature Communications.

[23]  Eric Betzig,et al.  Tiled Reconstruction Improves Structured Illumination Microscopy , 2020, bioRxiv.

[24]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[25]  Xiaocong Yuan,et al.  Fast structured illumination microscopy via deep learning , 2020 .

[26]  Abhay Patil,et al.  Learning Image Restoration without Clean Data , 2019 .

[27]  M. Sauer,et al.  Super-resolution microscopy demystified , 2019, Nature Cell Biology.

[28]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[29]  M. Gustafsson,et al.  Super-resolution 3D microscopy of live whole cells using structured illumination , 2011, Nature Methods.

[30]  Loic A. Royer,et al.  Content-aware image restoration: pushing the limits of fluorescence microscopy , 2018, Nature Methods.

[31]  F. Peyrin,et al.  A residual U-Net network with image prior for 3D image denoising , 2021, 2020 28th European Signal Processing Conference (EUSIPCO).

[32]  Atsushi Matsuda,et al.  Strategic and practical guidelines for successful structured illumination microscopy , 2017, Nature Protocols.

[33]  Reto Fiolka,et al.  Phase optimisation for structured illumination microscopy. , 2013, Optics express.

[34]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .

[35]  M. Davidson,et al.  Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination , 2012, Proceedings of the National Academy of Sciences.

[36]  M. Gustafsson,et al.  Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. , 2008, Biophysical journal.

[37]  Manuel Llinás,et al.  Dissecting the role of PfAP2-G in malaria gametocytogenesis , 2020, Nature Communications.

[38]  Wolfgang Hübner,et al.  Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ , 2016, Nature Communications.

[39]  Peng Xi,et al.  Structured Illumination Microscopy Image Reconstruction Algorithm , 2016, IEEE Journal of Selected Topics in Quantum Electronics.

[40]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[41]  Rainer Heintzmann,et al.  Super-Resolution Structured Illumination Microscopy. , 2017, Chemical reviews.

[42]  M. Gustafsson Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy , 2000, Journal of microscopy.