Accelerating iterative deconvolution and multiview fusion by orders of magnitude

We describe theoretical and practical advances in algorithm and software design, resulting in ten to several thousand-fold faster deconvolution and multiview fusion than previous methods. First, we adapt methods from medical imaging, showing that an unmatched back projector accelerates Richardson-Lucy deconvolution by at least 10-fold, in most cases requiring only a single iteration. Second, we show that improvements in 3D image-based registration with GPU processing result in speedups of 10-100-fold over CPU processing. Third, we show that deep learning can provide further accelerations, particularly for deconvolution with a spatially varying point spread function. We illustrate the power of our methods from the subcellular to millimeter spatial scale, on diverse samples including single cells, nematode and zebrafish embryos, and cleared mouse tissue. Finally, we show that our methods facilitate the use of new microscopes that improve spatial resolution, including dual-view cleared tissue light-sheet microscopy and reflective lattice light-sheet microscopy.

[1]  Cheuk Y. Tang,et al.  Mapping of Brain Activity by Automated Volume Analysis of Immediate Early Genes , 2016, Cell.

[2]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Stephan Preibisch,et al.  Efficient Bayesian-based multiview deconvolution , 2013, Nature Methods.

[4]  Melanie Grunwald,et al.  Foundations Of Image Science , 2016 .

[5]  Aljoscha Nern,et al.  Optimized tools for multicolor stochastic labeling reveal diverse stereotyped cell arrangements in the fly visual system , 2015, Proceedings of the National Academy of Sciences.

[6]  Shai Shaham,et al.  DEX-1 and DYF-7 Establish Sensory Dendrite Length by Anchoring Dendritic Tips during Cell Migration , 2009, Cell.

[7]  Eugene W. Myers,et al.  Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks , 2017, MICCAI.

[8]  Vladislav V. Verkhusha,et al.  A palette of fluorescent proteins optimized for diverse cellular environments , 2015, Nature Communications.

[9]  Abhishek Kumar,et al.  Simultaneous multiview capture and fusion improves spatial resolution in wide-field and light-sheet microscopy. , 2016, Optica.

[10]  Yicong Wu,et al.  Faster, sharper, and deeper: structured illumination microscopy for biological imaging , 2018, Nature Methods.

[11]  Jean Gautier,et al.  Early neural cell death: dying to become neurons. , 2004, Developmental biology.

[12]  Henry Pinkard,et al.  Advanced methods of microscope control using μManager software. , 2014, Journal of biological methods.

[13]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Lawrence D. True,et al.  Multi-immersion open-top light-sheet microscope for high-throughput imaging of cleared tissues , 2019 .

[17]  Stephan Preibisch,et al.  BigStitcher: Reconstructing high-resolution image datasets of cleared and expanded samples , 2018 .

[18]  Thomas Brox,et al.  Multiview Deblurring for 3-D Images from Light-Sheet-Based Fluorescence Microscopy , 2012, IEEE Transactions on Image Processing.

[19]  Hari Shroff,et al.  Richardson-Lucy deconvolution as a general tool for combining images with complementary strengths. , 2014, Chemphyschem : a European journal of chemical physics and physical chemistry.

[20]  Philipp J. Keller,et al.  Whole-animal functional and developmental imaging with isotropic spatial resolution , 2015, Nature Methods.

[21]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[22]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[23]  Johannes Schindelin,et al.  ImageJ Plugin CorrectBleach V2.0.2 , 2014 .

[24]  Gengsheng Lawrence Zeng,et al.  Unmatched projector/backprojector pairs in an iterative reconstruction algorithm , 2000, IEEE Transactions on Medical Imaging.

[25]  Anthony Santella,et al.  Isotropic Light-Sheet Microscopy and Automated Cell Lineage Analyses to Catalogue Caenorhabditis elegans Embryogenesis with Subcellular Resolution. , 2019, Journal of visualized experiments : JoVE.

[26]  Katie Bentley,et al.  Asymmetric division coordinates collective cell migration in angiogenesis , 2016, Nature Cell Biology.

[27]  Allan R. Jones,et al.  A robust and high-throughput Cre reporting and characterization system for the whole mouse brain , 2009, Nature Neuroscience.

[28]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[29]  Wesley R. Legant,et al.  Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution , 2014, Science.

[30]  E. Preston,et al.  The Bicoid Class Homeodomain Factors ceh-36/OTX and unc-30/PITX Cooperate in C. elegans Embryonic Progenitor Cells to Regulate Robust Development , 2015, PLoS genetics.

[31]  N. Renier,et al.  iDISCO: A Simple, Rapid Method to Immunolabel Large Tissue Samples for Volume Imaging , 2014, Cell.

[32]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[33]  Patrick J. La Riviere,et al.  Reflective imaging improves spatiotemporal resolution and collection efficiency in light sheet microscopy , 2017, Nature Communications.

[34]  Alexandra Bokinsky,et al.  Dual-view plane illumination microscopy for rapid and spatially isotropic imaging , 2014, Nature Protocols.

[35]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[36]  Justin Senseney,et al.  Spatially isotropic four-dimensional imaging with dual-view plane illumination microscopy , 2013, Nature Biotechnology.

[37]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[38]  E. Boyden,et al.  Simultaneous whole-animal 3D-imaging of neuronal activity using light-field microscopy , 2014, Nature Methods.

[39]  Nico Stuurman,et al.  Visualizing Calcium Flux in Freely Moving Nematode Embryos. , 2017, Biophysical journal.

[40]  Philippe Andrey,et al.  MorphoLibJ: integrated library and plugins for mathematical morphology with ImageJ , 2016, Bioinform..

[41]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[42]  Anthony Santella,et al.  Using Stage- and Slit-Scanning to Improve Contrast and Optical Sectioning in Dual-View Inverted Light Sheet Microscopy (diSPIM) , 2016, The Biological Bulletin.

[43]  Gaudenz Danuser,et al.  Light-sheet microscopy with isotropic, sub-micron resolution and solvent-independent large-scale imaging , 2019, bioRxiv.

[44]  John P Giannini,et al.  Single-shot super-resolution total internal reflection fluorescence microscopy , 2018, Nature Methods.

[45]  Masayuki Miura,et al.  Programmed cell death in neurodevelopment. , 2015, Developmental cell.

[46]  Stephan Saalfeld,et al.  Software for bead-based registration of selective plane illumination microscopy data , 2010, Nature Methods.

[47]  M. Gustafsson Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy , 2000, Journal of microscopy.

[48]  Hongkui Zeng,et al.  Multimodal cell type correspondence by intersectional mFISH in intact tissues , 2019, bioRxiv.

[49]  R. Waterston,et al.  Automated cell lineage tracing in Caenorhabditis elegans. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[50]  E. Fama,et al.  Migration , 2007 .

[51]  Andrew G. York,et al.  Instant super-resolution imaging in live cells and embryos via analog image processing , 2013, Nature Methods.

[52]  Anthony Santella,et al.  An In Toto Approach to Dissecting Cellular Interactions in Complex Tissues. , 2017, Developmental cell.

[53]  Darren Gilmour,et al.  Chemokine signaling mediates self-organizing tissue migration in the zebrafish lateral line. , 2006, Developmental cell.

[54]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.