A machine learning approach for online automated optimization of super-resolution optical microscopy

Traditional approaches for finding well-performing parameterizations of complex imaging systems, such as super-resolution microscopes rely on an extensive exploration phase over the illumination and acquisition settings, prior to the imaging task. This strategy suffers from several issues: it requires a large amount of parameter configurations to be evaluated, it leads to discrepancies between well-performing parameters in the exploration phase and imaging task, and it results in a waste of time and resources given that optimization and final imaging tasks are conducted separately. Here we show that a fully automated, machine learning-based system can conduct imaging parameter optimization toward a trade-off between several objectives, simultaneously to the imaging task. Its potential is highlighted on various imaging tasks, such as live-cell and multicolor imaging and multimodal optimization. This online optimization routine can be integrated to various imaging systems to increase accessibility, optimize performance and improve overall imaging quality.Complex imaging systems like super-resolution microscopes currently require laborious parameter optimization before imaging. Here, the authors present an imaging optimization framework based on machine learning that performs simultaneous parameter optimization to simplify this procedure for a wide range of imaging tasks.

[1]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[2]  T. Holak,et al.  Lifeact: a versatile marker to visualize F-actin , 2008, Nature Methods.

[3]  David Ogden,et al.  Laser photolysis of caged compounds at 405nm: Photochemical advantages, localisation, phototoxicity and methods for calibration , 2009, Journal of Neuroscience Methods.

[4]  K. Willig,et al.  In vivo mouse and live cell STED microscopy of neuronal actin plasticity using far-red emitting fluorescent proteins , 2017, Scientific Reports.

[5]  W. R. Thompson ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[6]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[7]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[8]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[9]  Bernardo L. Sabatini,et al.  Recombinant Probes for Visualizing Endogenous Synaptic Proteins in Living Neurons , 2013, Neuron.

[10]  Karel Svoboda,et al.  NMDA Receptor Subunit-Dependent [Ca2+] Signaling in Individual Hippocampal Dendritic Spines , 2005, The Journal of Neuroscience.

[11]  Paul De Koninck,et al.  Dissociated Hippocampal Cultures , 2009 .

[12]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[13]  Stefan R. Pulver,et al.  Ultra-sensitive fluorescent proteins for imaging neuronal activity , 2013, Nature.

[14]  Nello Cristianini,et al.  Finite-Time Analysis of Kernelised Contextual Bandits , 2013, UAI.

[15]  Joost van de Weijer,et al.  RankIQA: Learning from Rankings for No-Reference Image Quality Assessment , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Stefan W. Hell,et al.  Strong signal increase in STED fluorescence microscopy by imaging regions of subdiffraction extent , 2017, Proceedings of the National Academy of Sciences.

[17]  Alberto Diaspro,et al.  Evaluating image resolution in stimulated emission depletion microscopy , 2018 .

[18]  Amanda J Wright,et al.  Exploration of the optimisation algorithms used in the implementation of adaptive optics in confocal and multiphoton microscopy , 2005, Microscopy research and technique.

[19]  Ricardo Henriques,et al.  Superresolution imaging of HIV in infected cells with FlAsH-PALM , 2012, Proceedings of the National Academy of Sciences.

[20]  Audrey Durand,et al.  Learning to Become an Expert: Deep Networks Applied To Super-Resolution Microscopy , 2018, AAAI.

[21]  Alf Honigmann,et al.  Coaligned dual-channel STED nanoscopy and molecular diffusion analysis at 20 nm resolution. , 2013, Biophysical journal.

[22]  Christian Eggeling,et al.  Diffraction-unlimited all-optical imaging and writing with a photochromic GFP , 2011, Nature.

[23]  Stefan W. Hell,et al.  Adaptive-illumination STED nanoscopy , 2017, Proceedings of the National Academy of Sciences.

[24]  Christopher D. Saunter,et al.  Comparison of closed loop and sensorless adaptive optics in widefield optical microscopy , 2013 .

[25]  Jiang He,et al.  Developmental mechanism of the periodic membrane skeleton in axons , 2014, eLife.

[26]  Kevin W. Eliceiri,et al.  ImageJ2: ImageJ for the next generation of scientific image data , 2017, BMC Bioinformatics.

[27]  Sjoerd Stallinga,et al.  Measuring image resolution in optical nanoscopy , 2013, Nature Methods.

[28]  Stefan Jakobs,et al.  Novel red fluorophores with superior performance in STED microscopy , 2012, Optical Nanoscopy.

[29]  Jack P. Antel,et al.  Protocols for Neural Cell Culture edited by Sergey Fedoroff and Arlene Richardson, Humana Press, 1992. £51.00 (xiii + 178 pages) ISBN 0 89603 228 0 , 1993, Trends in Neurosciences.

[30]  J. Lisman,et al.  The molecular basis of CaMKII function in synaptic and behavioural memory , 2002, Nature Reviews Neuroscience.

[31]  G. Mourou,et al.  Smart microscope: an adaptive optics learning system for aberration correction in multiphoton confocal microscopy. , 2000, Optics letters.

[32]  R. Colbran,et al.  CaMKII: a molecular substrate for synaptic plasticity and memory. , 2014, Progress in molecular biology and translational science.

[33]  E. S. Gibson,et al.  Optogenetic Control of Synaptic Composition and Function , 2017, Neuron.

[34]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  S. van de Linde,et al.  Light-induced cell damage in live-cell super-resolution microscopy , 2015, Scientific Reports.

[36]  Jérôme Honnorat,et al.  Surface dynamics of GluN2B‐NMDA receptors controls plasticity of maturing glutamate synapses , 2014, The EMBO journal.

[37]  Joelle Pineau,et al.  Streaming kernel regression with provably adaptive mean, variance, and regularization , 2017, J. Mach. Learn. Res..

[38]  Quansen Sun,et al.  Single Image Super-Resolution Based on Deep Learning Features and Dictionary Model , 2017, IEEE Access.

[39]  Patrick Fox-Roberts,et al.  Local dimensionality determines imaging speed in localization microscopy , 2017, Nature Communications.

[40]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[42]  Kevin W. Eliceiri,et al.  ImageJ for the Next Generation of Scientific Image Data , 2019, Microscopy and Microanalysis.

[43]  Christian Eggeling,et al.  Red-emitting rhodamine dyes for fluorescence microscopy and nanoscopy. , 2010, Chemistry.

[44]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[45]  Andrew Gordon Wilson,et al.  Constant-Time Predictive Distributions for Gaussian Processes , 2018, ICML.

[46]  M Neal Waxham,et al.  A Mechanism for Ca2+/Calmodulin-Dependent Protein Kinase II Clustering at Synaptic and Nonsynaptic Sites Based on Self-Association , 2005, The Journal of Neuroscience.

[47]  X. Zhuang,et al.  Breaking the Diffraction Barrier: Super-Resolution Imaging of Cells , 2010, Cell.

[48]  M. Neil,et al.  Stimulated emission depletion microscopy with a supercontinuum source and fluorescence lifetime imaging. , 2008, Optics letters.

[49]  Christophe Leterrier,et al.  NanoJ-SQUIRREL: quantitative mapping and minimisation of super-resolution optical imaging artefacts , 2018, Nature Methods.

[50]  Kathleen D. Vohs,et al.  Making choices impairs subsequent self-control: A limited-resource account of decision making, self-regulation, and active initiative. , 2014 .

[51]  S. Hell,et al.  Fluorescence nanoscopy in cell biology , 2017, Nature Reviews Molecular Cell Biology.

[52]  Michel Meunier,et al.  Gold nanoparticle-assisted all optical localized stimulation and monitoring of Ca2+ signaling in neurons , 2016, Scientific Reports.

[53]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[54]  X. Zhuang,et al.  Actin, Spectrin, and Associated Proteins Form a Periodic Cytoskeletal Structure in Axons , 2013, Science.

[55]  Johannes E. Schindelin,et al.  Fiji: an open-source platform for biological-image analysis , 2012, Nature Methods.

[56]  D. Sculley,et al.  Large Scale Learning to Rank , 2009 .

[57]  S. Hell,et al.  Dual-label STED nanoscopy of living cells using photochromism. , 2011, Nano letters.

[58]  S. Hell,et al.  Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. , 1994, Optics letters.

[59]  Nicholas A. Frost,et al.  Multiple Spatial and Kinetic Subpopulations of CaMKII in Spines and Dendrites as Resolved by Single-Molecule Tracking PALM , 2014, The Journal of Neuroscience.

[60]  H. Robbins Some aspects of the sequential design of experiments , 1952 .

[61]  Ian M. Dobbie,et al.  Imaging cellular structures in super-resolution with SIM, STED and Localisation Microscopy: A practical comparison , 2016, Scientific Reports.

[62]  J. Elf,et al.  Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes , 2016, Science.

[63]  Flavie Lavoie-Cardinal,et al.  Two-color RESOLFT nanoscopy with green and red fluorescent photochromic proteins. , 2014, Chemphyschem : a European journal of chemical physics and physical chemistry.

[64]  M. Beck,et al.  Fourier ring correlation as a resolution criterion for super-resolution microscopy. , 2013, Journal of structural biology.

[65]  Rémi Munos,et al.  Thompson Sampling: An Asymptotically Optimal Finite-Time Analysis , 2012, ALT.

[66]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[67]  U Valentin Nägerl,et al.  Two-color STED microscopy of living synapses using a single laser-beam pair. , 2011, Biophysical journal.

[68]  M. Ehrgott Vilfredo Pareto and multi-objective optimization , 2012 .

[69]  T. L. Lai Andherbertrobbins Asymptotically Efficient Adaptive Allocation Rules , 1985 .

[70]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.