A deep learning-based stripe self-correction method for stitched microscopic images

The stitched fluorescence microscope images inevitably exist in various types of stripes or artifacts caused by uncertain factors such as optical devices or specimens, which severely affects the image quality and downstream quantitative analysis. In this paper, we present a deep learning-based Stripe Self-Correction method, so-called SSCOR. Specifically, we propose a proximity sampling scheme and adversarial reciprocal self-training paradigm that enable SSCOR to utilize stripe-free patches sampled from the stitched microscope image itself to correct their adjacent stripe patches. Comparing to off-the-shelf approaches, SSCOR can not only adaptively correct non-uniform, oblique, and grid stripes, but also remove scanning, bubble, and out-of-focus artifacts, achieving the state-of-the-art performance across different imaging conditions and modalities. Moreover, SSCOR does not require any physical parameter estimation, patch-wise manual annotation, and raw stitched information in the correction process, which provides an intelligent image quality optimization solution for microscopist or even microscope company, and facilitates more precise biomedical applications for researchers.

[1]  Shuoyu Xu,et al.  Resection-inspired histopathological diagnosis of cerebral cavernous malformations using quantitative multiphoton microscopy , 2022, Theranostics.

[2]  S. Duan,et al.  Olfactory sensory experience regulates gliomagenesis via neuronal IGF1 , 2022, Nature.

[3]  J. Qu,et al.  Deep learning autofluorescence-harmonic microscopy , 2022, Light: Science & Applications.

[4]  Enhua Wu,et al.  GhostNets on Heterogeneous Devices via Cheap Operations , 2022, International Journal of Computer Vision.

[5]  Chuan Wang,et al.  Computer-assisted quantification of tumor-associated collagen signatures to improve the prognosis prediction of breast cancer , 2021, BMC Medicine.

[6]  Jiubin Tan,et al.  Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy , 2021, Nature Biotechnology.

[7]  M. Unser,et al.  DeepImageJ: A user-friendly environment to run deep learning models in ImageJ , 2021, Nature Methods.

[8]  Hongbin Han,et al.  Correction of uneven illumination in color microscopic image based on fully convolutional network. , 2021, Optics express.

[9]  Ki Hean Kim,et al.  Deep learning enables reference-free isotropic super-resolution for volumetric fluorescence microscopy , 2021, Nature Communications.

[10]  S. Boppart,et al.  Label-Free Deep Profiling of the Tumor Microenvironment , 2021, Cancer Research.

[11]  Qionghai Dai,et al.  Unsupervised content-preserving transformation for optical microscopy , 2021, Light: Science & Applications.

[12]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[13]  Alec Radford,et al.  Zero-Shot Text-to-Image Generation , 2021, ICML.

[14]  Xu Jia,et al.  Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Dacheng Tao,et al.  Searching for Low-Bit Weights in Quantized Neural Networks , 2020, NeurIPS.

[16]  Aydogan Ozcan,et al.  Deep learning-based transformation of H&E stained tissues into special stains , 2020, Nature Communications.

[17]  G. Schackert,et al.  Label-free multiphoton microscopy as a tool to investigate alterations of cerebral aneurysms , 2020, Scientific Reports.

[18]  Angeliki Pollatou,et al.  An automated method for removal of striping artifacts in fluorescent whole-slide microscopy , 2020, Journal of Neuroscience Methods.

[19]  Qionghai Dai,et al.  High-speed, multi-modal, label-free imaging of pathological slices with a Bessel beam. , 2020, Biomedical optics express.

[20]  Chen Change Loy,et al.  Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Liwei Liu,et al.  Label-free whole-colony imaging and metabolic analysis of metastatic pancreatic cancer by an autoregulating flexible optical system , 2020, Theranostics.

[22]  Saurabh Sinha,et al.  Real-time intraoperative diagnosis by deep neural network driven multiphoton virtual histology , 2019, npj Precision Oncology.

[23]  Qionghai Dai,et al.  Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution , 2019, Nature Photonics.

[24]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[25]  David Kaeli,et al.  Exploiting Adaptive Data Compression to Improve Performance and Energy-Efficiency of Compute Workloads in Multi-GPU Systems , 2019, 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[26]  J. Livet,et al.  Multicolor multiscale brain imaging with chromatic multiphoton serial microscopy , 2019, Nature Communications.

[27]  J. D. de Munck,et al.  Quantitative Third Harmonic Generation Microscopy for Assessment of Glioma in Human Brain Tissue , 2019, Advanced science.

[28]  Chi-Wing Fu,et al.  Mask-ShadowGAN: Learning to Remove Shadows From Unpaired Data , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[29]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2019, Nature Biomedical Engineering.

[30]  Jin Tae Kwak,et al.  Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images , 2018, Medical Image Anal..

[31]  Loic A. Royer,et al.  Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction , 2018, Nature Methods.

[32]  Joanne Li,et al.  High-speed imaging of transient metabolic dynamics using two-photon fluorescence lifetime imaging microscopy. , 2018, Optica.

[33]  Reza Azimi,et al.  PowerCoord: A Coordinated Power Capping Controller for Multi-CPU/GPU Servers , 2018, 2018 Ninth International Green and Sustainable Computing Conference (IGSC).

[34]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[35]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[36]  Minbiao Ji,et al.  Rapid, large-scale stimulated Raman histology with strip mosaicing and dual-phase detection. , 2018, Biomedical optics express.

[37]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Nassir Navab,et al.  A BaSiC tool for background and shading correction of optical microscopy images , 2017, Nature Communications.

[39]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Shaoqun Zeng,et al.  High-throughput dual-colour precision imaging for brain-wide connectome with cytoarchitectonic landmarks at the cellular level , 2016, Nature Communications.

[41]  Isaiah Norton,et al.  Label-Free Neurosurgical Pathology with Stimulated Raman Imaging. , 2016, Cancer research.

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Yunpeng Li,et al.  CIDRE: an illumination-correction method for optical microscopy , 2015, Nature Methods.

[44]  Satwik Rajaram,et al.  A simple image correction method for high-throughput microscopy , 2014, Nature Methods.

[45]  Jeremy D. Schmahmann,et al.  A Proposal for a Coordinated Effort for the Determination of Brainwide Neuroanatomical Connectivity in Model Organisms at a Mesoscopic Scale , 2009, PLoS Comput. Biol..

[46]  Wataru Takeuchi,et al.  Stripe Noise Reduction in MODIS Data by Combining Histogram Matching With Facet Filter , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[48]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[49]  Yuming Liu,et al.  Methods for Quantifying Fibrillar Collagen Alignment. , 2017, Methods in molecular biology.