High-Fidelity Reconstruction of Structured Illumination Microscopy by an Amplitude-Phase Channel Attention Network With Multitemporal Information

Structured illumination microscopy (SIM) has been a popular method for live-cell super-resolution (SR) imaging due to its excellent photon efficiency. However, SIM is always marred by artifacts in the reconstructed SR images. To address this problem, we develop an end-to-end amplitude-phase channel attention network (APCAN) for SIM reconstruction based on a priori frequency and temporal knowledge. The APCAN reinforces both amplitude and phase information of the raw images to guide network reconstruction and attains SR images with fewer artifacts. Moreover, inspired by the continuity knowledge in the traditional method, we design a temporal processing module in APCAN to utilize multiple time points data. Trained with the video dataset imaged from our setup, APCAN can reconstruct an artifact-minimized SR image, achieving a reduction of 38% in reconstruction errors. Finally, we demonstrate that the APCAN reconstructed images have great resistance to noise and photobleaching, achieving the best fidelity among all methods tested.

[1]  Zach Burns,et al.  Untrained, physics-informed neural networks for structured illumination microscopy , 2022, Optics express.

[2]  Zhaojun Wang,et al.  High-speed image reconstruction for optically sectioned, super-resolution structured illumination microscopy , 2022, Advanced Photonics.

[3]  Edward N. Ward,et al.  Spatio-temporal Vision Transformer for Super-resolution Microscopy , 2022, ArXiv.

[4]  Syed Waqas Zamir,et al.  Restormer: Efficient Transformer for High-Resolution Image Restoration , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Xi Cheng,et al.  Fast and Lightweight Network for Single Frame Structured Illumination Microscopy Super-Resolution , 2021, IEEE Transactions on Instrumentation and Measurement.

[6]  S. Du,et al.  Deep-Learning Super-Resolution Microscopy Reveals Nanometer-Scale Intracellular Dynamics at the Millisecond Temporal Resolution , 2021, bioRxiv.

[7]  Romain F. Laine,et al.  Avoiding a replication crisis in deep-learning-based bioimage analysis , 2021, Nature Methods.

[8]  Luc Van Gool,et al.  SwinIR: Image Restoration Using Swin Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[9]  Yonghong Tian,et al.  Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Jindong Tian,et al.  Single-shot structured illumination microscopy , 2021, 2107.05930.

[11]  Qionghai Dai,et al.  3D Structured Illumination Microscopy via Channel Attention Generative Adversarial Network , 2021, IEEE Journal of Selected Topics in Quantum Electronics.

[12]  P. Kner,et al.  Subcellular three-dimensional imaging deep through multicellular thick samples by structured illumination microscopy and adaptive optics , 2021, Nature Communications.

[13]  Baocai Yin,et al.  A Two-Stage Attentive Network for Single Image Super-Resolution , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Yuguo Tang,et al.  High-fidelity structured illumination microscopy by point-spread-function engineering , 2021, Light, science & applications.

[15]  C. W. Hagen,et al.  Structured illumination microscopy with noise-controlled image reconstructions , 2021, Nature Methods.

[16]  Edward N. Ward,et al.  ML-SIM: universal reconstruction of structured illumination microscopy images using transfer learning , 2021, Biomedical optics express.

[17]  Qionghai Dai,et al.  Evaluation and development of deep neural networks for image super-resolution in optical microscopy , 2021, Nature Methods.

[18]  Chen Change Loy,et al.  Focal Frequency Loss for Image Reconstruction and Synthesis , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  N. Brockdorff,et al.  Time-resolved structured illumination microscopy reveals key principles of Xist RNA spreading , 2020, Science.

[20]  Tung-Cheng Wang,et al.  Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images , 2020, bioRxiv.

[21]  Xiaocong Yuan,et al.  Fast structured illumination microscopy via deep learning , 2020 .

[22]  Fenqiang Zhao,et al.  Deep learning enables structured illumination microscopy with low light levels and enhanced speed , 2019, Nature Communications.

[23]  A. Descloux,et al.  Parameter-free image resolution estimation based on decorrelation analysis , 2019, Nature Methods.

[24]  Liangyi Chen,et al.  A protocol for structured illumination microscopy with minimal reconstruction artifacts , 2019, Biophysics Reports.

[25]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[26]  J. Lippincott-Schwartz,et al.  Visualizing Intracellular Organelle and Cytoskeletal Interactions at Nanoscale Resolution on Millisecond Timescales , 2018, Cell.

[27]  Liangyi Chen,et al.  Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy , 2018, Nature Biotechnology.

[28]  Rainer Heintzmann,et al.  Super-Resolution Structured Illumination Microscopy. , 2017, Chemical reviews.

[29]  Atsushi Matsuda,et al.  Strategic and practical guidelines for successful structured illumination microscopy , 2017, Nature Protocols.

[30]  Wolfgang Hübner,et al.  Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ , 2016, Nature Communications.

[31]  M. Davidson,et al.  Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics , 2015, Science.

[32]  Jie Yin,et al.  Image reconstruction for structured-illumination microscopy with low signal level. , 2014, Optics express.

[33]  M. Gustafsson,et al.  Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. , 2008, Biophysical journal.

[34]  M. Gustafsson.  Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy , 2000, Journal of microscopy.

[35]  Shaowei Jiang,et al.  Accelerated Phase Shifting for Structured Illumination Microscopy Based on Deep Learning , 2021, IEEE Transactions on Computational Imaging.