Improving axial resolution in SIM using deep learning

Structured Illumination Microscopy is a widespread methodology to image live and fixed biological structures smaller than the diffraction limits of conventional optical microscopy. Using recent advances in image up-scaling through deep learning models, we demonstrate a method to reconstruct 3D SIM image stacks with twice the axial resolution attainable through conventional SIM reconstructions. We further evaluate our method for robustness to noise & generalisability to varying observed specimens, and discuss potential adaptions of the method to further improvements in resolution.

[1]  Jinbo Xu,et al.  Inferential modeling of 3D chromatin structure , 2015, Nucleic acids research.

[2]  Hari Shroff,et al.  Two-photon excitation improves multifocal structured illumination microscopy in thick scattering tissue , 2014, Proceedings of the National Academy of Sciences.

[3]  Liangyi Chen,et al.  A protocol for structured illumination microscopy with minimal reconstruction artifacts , 2019, Biophysics Reports.

[4]  Steven C. H. Hoi,et al.  Deep Learning for Image Super-Resolution: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Fenqiang Zhao,et al.  Deep learning enables structured illumination microscopy with low light levels and enhanced speed , 2019, Nature Communications.

[6]  Ian M. Dobbie,et al.  Imaging cellular structures in super-resolution with SIM, STED and Localisation Microscopy: A practical comparison , 2016, Scientific Reports.

[7]  M. Gustafsson.  Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy , 2000, Journal of microscopy.

[8]  Clemens F. Kaminski,et al.  A Guide to Structured Illumination TIRF Microscopy at High Speed with Multiple Colors , 2016, Journal of visualized experiments : JoVE.

[9]  Loic A. Royer,et al.  Content-aware image restoration: pushing the limits of fluorescence microscopy , 2018, Nature Methods.

[10]  P. Carlton,et al.  Interlock Formation and Coiling of Meiotic Chromosome Axes During Synapsis , 2009, Genetics.

[11]  R. Osellame,et al.  High-throughput 3D imaging of single cells with light-sheet fluorescence microscopy on chip. , 2020, Biomedical Optics Express.

[12]  Benjamin B. Machta,et al.  Correlation Functions Quantify Super-Resolution Images and Estimate Apparent Clustering Due to Over-Counting , 2011, PloS one.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[15]  Pietro Lio,et al.  ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images , 2020, ArXiv.

[16]  Xiaocong Yuan,et al.  Fast structured illumination microscopy via deep learning , 2020 .

[17]  Michael Unser,et al.  Publisher Correction: Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software , 2019, Nature Methods.

[18]  R. London,et al.  Signal, noise and resolution in linear and nonlinear structured‐illumination microscopy , 2018, Journal of microscopy.

[19]  M. Davidson,et al.  Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination , 2012, Proceedings of the National Academy of Sciences.

[20]  Rainer Heintzmann,et al.  Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating , 1999, European Conference on Biomedical Optics.

[21]  M. Gustafsson,et al.  Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. , 2008, Biophysical journal.

[22]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[23]  Hari Shroff,et al.  Resolution Doubling in Live, Multicellular Organisms via Multifocal Structured Illumination Microscopy , 2012, Nature Methods.

[24]  Eric Betzig,et al.  Dynamic super-resolution structured illumination imaging in the living brain , 2019, Proceedings of the National Academy of Sciences.