Deep learning enables structured illumination microscopy with low light levels and enhanced speed

Using deep learning to augment structured illumination microscopy (SIM), we obtained a fivefold reduction in the number of raw images required for super-resolution SIM, and generated images under extreme low light conditions (100X fewer photons). We validated the performance of deep neural networks on different cellular structures and achieved multi-color, live-cell super-resolution imaging with greatly reduced photobleaching.

[1]  Victor Perez,et al.  Optimal 2D-SIM reconstruction by two filtering steps with Richardson-Lucy deconvolution , 2016, Scientific Reports.

[2]  A. Descloux,et al.  Parameter-free image resolution estimation based on decorrelation analysis , 2019, Nature Methods.

[3]  Maria Smedh,et al.  Successful optimization of reconstruction parameters in structured illumination microscopy – a practical guide , 2018, bioRxiv.

[4]  Alejandro F. Frangi,et al.  Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015 , 2015, Lecture Notes in Computer Science.

[5]  Sahil Malik,et al.  What you Should Know , 2018, Deploy, Secure and Manage Azure Functions.

[6]  Ricardo Henriques,et al.  Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations , 2016, Nature Communications.

[7]  Thomas Brox,et al.  Author Correction: U-Net: deep learning for cell counting, detection, and morphometry , 2019, Nature Methods.

[8]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 , 2015, Lecture Notes in Computer Science.

[9]  William Graf,et al.  Deep learning for cellular image analysis , 2019, Nature Methods.

[10]  P. Xi,et al.  Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy , 2018, Nature Biotechnology.

[11]  J. Lippincott-Schwartz,et al.  Visualizing Intracellular Organelle and Cytoskeletal Interactions at Nanoscale Resolution on Millisecond Timescales , 2018, Cell.

[12]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[15]  Christophe Leterrier,et al.  NanoJ-SQUIRREL: quantitative mapping and minimisation of super-resolution optical imaging artefacts , 2017, Nature Methods.

[16]  M. Gustafsson.  Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy , 2000, Journal of microscopy.

[17]  Ricardo Henriques,et al.  Artificial intelligence for microscopy: what you should know. , 2019, Biochemical Society transactions.

[18]  Loic A. Royer,et al.  Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction , 2018, Nature Methods.

[19]  Thomas Huser,et al.  Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction , 2019, Nature Communications.

[20]  Holly M. Hutchins,et al.  Training Transfer: An Integrative Literature Review , 2007 .

[21]  Loic A. Royer,et al.  Content-aware image restoration: pushing the limits of fluorescence microscopy , 2018, Nature Methods.

[22]  Kun Zhao,et al.  A Frequency Domain SIM Reconstruction Algorithm Using Reduced Number of Images , 2018, IEEE Transactions on Image Processing.

[23]  Ian M. Dobbie,et al.  SIMcheck: a toolbox for successful super-resolution SIM imaging , 2015 .