Super-resolution fluorescence microscopy based on physical models

This thesis introduces a collection of physics-based methods for super-resolution in optical microscopy. The core of these methods constitutes a framework for 3-D localization of single fluorescent molecules. Localization is formulated as a parameter estimation problem relying on a physically accurate model of the system's point spread function (PSF). In a similar approach, methods for fitting PSF models to experimental observations and for extended-depth-of-field imaging are proposed. Imaging of individual fluorophores within densely labeled samples has become possible with the discovery of dyes that can be photo-activated or switched between fluorescent and dark states. A fluorophore can be localized from its image with nanometer-scale accuracy, through fitting with an appropriate image function. This concept forms the basis of fluorescence localization microscopy (FLM) techniques such as photo-activated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM), which rely on Gaussian fitting to perform the localization. Whereas the image generated by a single fluorophore corresponds to a section of the microscope's point spread function, only the in-focus section of the latter is well approximated by a Gaussian. Consequently, applications of FLM have for the most part been limited to 2-D imaging of thin specimen layers. In the first section of the thesis, it is shown that localization can be extended to 3-D without loss in accuracy by relying on a physically accurate image formation model in place of a Gaussian approximation. A key aspect of physically realistic models lies in their incorporation of aberrations that arise either as a consequence of mismatched refractive indices between the layers of the sample setup, or as an effect of experimental settings that deviate from the design conditions of the system. Under typical experimental conditions, these aberrations change as a function of sample depth, inducing axial shift-variance in the PSF. This property is exploited in a maximum-likelihood framework for 3-D localization of single fluorophores. Due to the shift-variance of the PSF, the axial position of a fluorophore is uniquely encoded in its diffraction pattern, and can be estimated from a single acquisition with nanometer-scale accuracy. Fluroescent molecules that remain fixed during image acquisition produce a diffraction pattern that is highly characteristic of the orientation of the fluorophore's underlying electromagnetic dipole. The localization is thus extended to incorporate the estimation of this 3-D orientation. It is shown that image formation for dipoles can be represented through a combination of six functions, based on which a 3-D steerable filter for orientation estimation and localization is derived. Experimental results demonstrate the feasibility of joint position and orientation estimation with accuracies of the order of one nanometer and one degree, respectively. Theoretical limits on localization accuracy for these methods are established in a statistical analysis based on Cramer-Rao bounds. A noise model for fluorescence microscopy based on a shifted Poisson distribution is proposed. In these localization methods, the aberration parameters of the PSF are assumed to be known. However, state-of-the-art PSF models depend on a large number of parameters that are generally difficult to determine experimentally with sufficient accuracy, which may limit their use in localization and deconvolution applications. A fitting algorithm analogous to localization is proposed; it is based on a simplified PSF model and shown to accurately reproduce the behavior of shift-variant experimental PSFs. Finally, it is shown that these algorithms can be adapted to more complex experiments, such as imaging of thick samples in brightfield microscopy. An extended-depth-of-field algorithm for the fusion of in-focus information from frames acquired at different focal positions is presented; it relies on a spline representation of the sample surface and as a result yields a continuous and super-resolved topography of the specimen.

[1]  D. Anderson,et al.  Algorithms for minimization without derivatives , 1974 .

[2]  Dan McLachlan,et al.  Extreme Focal Depth in Microscopy , 1964 .

[3]  George Biddell Airy,et al.  On the diffraction of an object-glass with circular aperture , 1835 .

[4]  Jörg Enderlein,et al.  Image Analysis of Defocused Single-Molecule Images for Three-Dimensional Molecule Orientation Studies , 2004 .

[5]  Paul R. Selvin,et al.  Myosin V Walks Hand-Over-Hand: Single Fluorophore Imaging with 1.5-nm Localization , 2003, Science.

[6]  Xiaobo Zhou,et al.  Informatics challenges of high-throughput microscopy , 2006, IEEE Signal Processing Magazine.

[7]  Sheyum Syed,et al.  Defocused orientation and position imaging (DOPI) of myosin V. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[8]  H. P. Kao,et al.  Tracking of single fluorescent particles in three dimensions: use of cylindrical optics to encode particle position. , 1994, Biophysical journal.

[9]  M. Minsky Memoir on inventing the confocal scanning microscope , 1988 .

[10]  Michael Unser,et al.  Elastic registration of biological images using vector-spline regularization , 2005, IEEE Transactions on Biomedical Engineering.

[11]  Mark Bates,et al.  Short-range spectroscopic ruler based on a single-molecule optical switch. , 2005, Physical review letters.

[12]  J. Swedlow,et al.  A workingperson's guide to deconvolution in light microscopy. , 2001, BioTechniques.

[13]  A. Cramer-Rao lower bounds on the performance of charge-coupled-device optical position estimators , 1985 .

[14]  T. Kues,et al.  Imaging and tracking of single GFP molecules in solution. , 2000, Biophysical journal.

[15]  Michael Unser,et al.  User‐friendly semiautomated assembly of accurate image mosaics in microscopy , 2007, Microscopy research and technique.

[16]  S. Gibson,et al.  Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[17]  Michael Unser,et al.  Automatic tracking of individual fluorescence particles: application to the study of chromosome dynamics , 2005, IEEE Transactions on Image Processing.

[18]  Akram Aldroubi,et al.  B-SPLINE SIGNAL PROCESSING: PART I-THEORY , 1993 .

[19]  Juškaitis,et al.  The measurement of the amplitude point spread function of microscope objective lenses , 1998 .

[20]  M. Gu,et al.  Advanced Optical Imaging Theory , 1999 .

[21]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[22]  E. Wolf,et al.  Principles of Optics (7th Ed) , 1999 .

[23]  P. Sorger,et al.  Automatic fluorescent tag localization II: improvement in super‐resolution by relative tracking , 2003, Journal of microscopy.

[24]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[25]  P. Debye,et al.  Das Verhalten von Lichtwellen in der Nähe eines Brennpunktes oder einer Brennlinie , 1909 .

[26]  M. Gustafsson,et al.  S: Widefield Light Microscopy with 100-nm-scale Resolution in Three Dimensions , 2007 .

[27]  Jean-Christophe Olivo-Marin,et al.  On the digital trail of mobile cells , 2006, IEEE Signal Processing Magazine.

[28]  H. H. Hopkins The frequency response of a defocused optical system , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[29]  Travis J Gould,et al.  Nanoscale imaging of molecular positions and anisotropies , 2008, Nature Methods.

[30]  A. Einstein Über einen die Erzeugung und Verwandlung des Lichtes betreffenden heuristischen Gesichtspunkt [AdP 17, 132 (1905)] , 2005, Annalen der Physik.

[31]  J. Eisinger,et al.  The orientational freedom of molecular probes. The orientation factor in intramolecular energy transfer. , 1979, Biophysical journal.

[32]  Robert M. Dickson,et al.  Imaging Three-Dimensional Single Molecule Orientations , 1999 .

[33]  A. Bradley,et al.  A One-Pass Extended Depth of Field Algorithm Based on the Over-Complete Discrete Wavelet Transform , 2004 .

[34]  Stefan W. Hell,et al.  Equivalence of the Huygens–Fresnel and Debye approach for the calculation of high aperture point‐spread functions in the presence of refractive index mismatch , 1999 .

[35]  D. Axelrod Cell-substrate contacts illuminated by total internal reflection fluorescence , 1981, The Journal of cell biology.

[36]  Ulrich Kubitscheck Single Protein Molecules Visualized and Tracked in the Interior of Eukaryotic Cells , 2002 .

[37]  M. Unser,et al.  A maximum-likelihood formalism for sub-resolution axial localization of fluorescent nanoparticles. , 2005, Optics express.

[38]  H. H. Hopkins,et al.  The concept of partial coherence in optics , 1951, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[39]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[40]  E. Abbe Beiträge zur Theorie des Mikroskops und der mikroskopischen Wahrnehmung , 1873 .

[41]  Enrico Gratton,et al.  3-D particle tracking in a two-photon microscope: application to the study of molecular dynamics in cells. , 2005, Biophysical journal.

[42]  M K Cheezum,et al.  Quantitative comparison of algorithms for tracking single fluorescent particles. , 2001, Biophysical journal.

[43]  T. Maiman Stimulated Optical Radiation in Ruby , 1960, Nature.

[44]  Gabriel Thomas,et al.  All-in-Focus Imaging Using a Series of Images on Different Focal Planes , 2005, ICIAR.

[45]  G. W. Stroke,et al.  White-light Three Dimensional Microscopy using Multiple-image Storing and Decoding , 1971, Nature.

[46]  Dimitri Van De Ville,et al.  On the feasibility of axial tracking of a fluorescent nanoparticle using a defocusing model , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[47]  Mats Ulfendahl,et al.  Image-adaptive deconvolution for three-dimensional deep biological imaging. , 2003, Biophysical journal.

[48]  N. Bobroff Position measurement with a resolution and noise‐limited instrument , 1986 .

[49]  G. E. Smith,et al.  Charge coupled semiconductor devices , 1970, Bell Syst. Tech. J..

[50]  L. J. Thomas,et al.  Artifacts in computational optical-sectioning microscopy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[51]  D. Axelrod,et al.  Fluorescence emission at dielectric and metal-film interfaces , 1987 .

[52]  Joshua W Shaevitz,et al.  Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[53]  M. Labouesse,et al.  Assembly of C. elegans apical junctions involves positioning and compaction by LET-413 and protein aggregation by the MAGUK protein DLG-1. , 2001, Journal of cell science.

[54]  Stefan W. Hell,et al.  Fundamental improvement of resolution with a 4Pi-confocal fluorescence microscope using two-photon excitation , 1992 .

[55]  D. Agard,et al.  The use of a charge-coupled device for quantitative optical microscopy of biological structures. , 1987, Science.

[56]  David A. Agard,et al.  Three-dimensional architecture of a polytene nucleus , 1983, Nature.

[57]  R Y Tsien,et al.  Specific covalent labeling of recombinant protein molecules inside live cells. , 1998, Science.

[58]  J. Lippincott-Schwartz,et al.  Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure , 2009, Proceedings of the National Academy of Sciences.

[59]  Stefan W. Hell,et al.  Aberrations in confocal and multi-photon fluorescence microscopy induced by refractive index mismatch , 2006 .

[60]  Rahul Roy,et al.  A practical guide to single-molecule FRET , 2008, Nature Methods.

[61]  T. Stearns,et al.  The green revolution , 1995 .

[62]  Jürgen Köhler,et al.  3-Dimensional super-resolution by spectrally selective imaging , 1998 .

[63]  M. Meier,et al.  Optical sectioning in wide-field microscopy obtained by dynamic structured light illumination and detection based on a smart pixel detector array. , 2003, Optics letters.

[64]  Colin J. R. Sheppard,et al.  Optical microscopy with extended depth of field , 1983, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[65]  Aleksandra Pizurica,et al.  Extending the Depth of Field in Microscopy Through Curvelet-Based Frequency-Adaptive Image Fusion , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[66]  Alexandr Jonás,et al.  Three-dimensional tracking of fluorescent nanoparticles with subnanometer precision by use of off-focus imaging. , 2003, Optics letters.

[67]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[68]  J. Sibarita Deconvolution microscopy. , 2005, Advances in biochemical engineering/biotechnology.

[69]  S. Gibson,et al.  Diffraction by a circular aperture as a model for three-dimensional optical microscopy. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[70]  O. Shimomura,et al.  Extraction, purification and properties of aequorin, a bioluminescent protein from the luminous hydromedusan, Aequorea. , 1962, Journal of cellular and comparative physiology.

[71]  Chrysanthe Preza,et al.  Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[72]  Th. Förster Zwischenmolekulare Energiewanderung und Fluoreszenz , 1948 .

[73]  Ian Parker,et al.  Multiphoton-evoked color change of DsRed as an optical highlighter for cellular and subcellular labeling , 2001, Nature Biotechnology.

[74]  Konstantin A Lukyanov,et al.  Photoswitchable cyan fluorescent protein for protein tracking , 2004, Nature Biotechnology.

[75]  Jörg Enderlein,et al.  Orientation imaging of single molecules by wide-field epifluorescence microscopy , 2003 .

[76]  Y. Ichioka,et al.  Digitized optical microscopy with extended depth of field. , 1989, Applied optics.

[77]  A. Schawlow,et al.  Infrared and optical masers , 1958 .

[78]  Jorge Ojeda-Castañeda,et al.  Asymmetric phase masks for extended depth of field. , 2004, Applied optics.

[79]  E. Wolf,et al.  Electromagnetic diffraction in optical systems - I. An integral representation of the image field , 1959, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[80]  Michael Liebling,et al.  Four-dimensional cardiac imaging in living embryos via postacquisition synchronization of nongated slice sequences. , 2005, Journal of biomedical optics.

[81]  Mark Bates,et al.  Three-Dimensional Super-Resolution Imaging by Stochastic Optical Reconstruction Microscopy , 2008, Science.

[82]  Michael D. Mason,et al.  Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. , 2006, Biophysical journal.

[83]  Michael W. Davidson,et al.  Dual-color superresolution imaging of genetically expressed probes within individual adhesion complexes , 2007, Proceedings of the National Academy of Sciences.

[84]  Jean-Christophe Olivo-Marin,et al.  Extraction of spots in biological images using multiscale products , 2002, Pattern Recognit..

[85]  Rainer Heintzmann,et al.  Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating , 1999, European Conference on Biomedical Optics.

[86]  J. Lippincott-Schwartz,et al.  Imaging Intracellular Fluorescent Proteins at Nanometer Resolution , 2006, Science.

[87]  D. Axelrod Total internal reflection fluorescence microscopy in cell biology. , 2003, Methods in enzymology.

[88]  F. Tsuji,et al.  Aequorea green fluorescent protein , 1994, FEBS letters.

[89]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[90]  R J Pieper,et al.  Image processing for extended depth of field. , 1983, Applied optics.

[91]  Y Ichioka,et al.  Digital composition of images with increased depth of focus considering depth information. , 1985, Applied optics.

[92]  Peter J. Burt,et al.  Enhanced image capture through fusion , 1993, 1993 (4th) International Conference on Computer Vision.

[93]  Lord Rayleigh,et al.  On the Theory of Optical Images, with Special Reference to the Microscope , 1903 .

[94]  R Kompfner,et al.  Resonant scanning optical microscope. , 1978, Applied optics.

[95]  A Diaspro,et al.  Single‐pinhole confocal imaging of sub‐resolution sparse objects using experimental point spread function and image restoration , 2000, Microscopy research and technique.

[96]  A. Marathay,et al.  On the usual approximation used in the Rayleigh-Sommerfeld diffraction theory. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[97]  J Boutet de Monvel,et al.  Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ. , 2001, Biophysical journal.

[98]  G Indebetouw,et al.  Imaging with Fresnel zone pupil masks: extended depth of field. , 1984, Applied optics.

[99]  J. Conchello,et al.  Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[100]  Vannary Meas-Yedid,et al.  Segmentation and tracking of migrating cells in videomicroscopy with parametric active contours: a tool for cell-based drug testing , 2002, IEEE Transactions on Medical Imaging.

[101]  S. Hell,et al.  Spherical nanosized focal spot unravels the interior of cells , 2008, Nature Methods.

[102]  Michael Unser,et al.  Super-resolution orientation estimation and localization of fluorescent dipoles using 3-D steerable filters. , 2009, Optics express.

[103]  W T Cathey,et al.  Extended depth of field and aberration control for inexpensive digital microscope systems. , 1999, Optics express.

[104]  S. Gambhir,et al.  Quantum Dots for Live Cells, in Vivo Imaging, and Diagnostics , 2005, Science.

[105]  S. Hell,et al.  Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. , 1994, Optics letters.

[106]  E. Wolf,et al.  Electromagnetic diffraction in optical systems, II. Structure of the image field in an aplanatic system , 1959, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[107]  Peter Török,et al.  Electromagnetic diffraction of light focused through a planar interface between materials of mismatched refractive indices: an integral representation , 1995 .

[108]  G. W. Ford,et al.  Electromagnetic interactions of molecules with metal surfaces , 1984 .

[109]  J. Zerubia,et al.  Gaussian approximations of fluorescence microscope point-spread function models. , 2007, Applied optics.

[110]  J. Lippincott-Schwartz,et al.  Development and Use of Fluorescent Protein Markers in Living Cells , 2003, Science.

[111]  Ivor W. Tsang,et al.  Fusing images with different focuses using support vector machines , 2004, IEEE Transactions on Neural Networks.

[112]  R. Pepperkok,et al.  Spectral imaging and its applications in live cell microscopy , 2003, FEBS letters.

[113]  Weidong Yang,et al.  Linearly Polarized Emission from Colloidal Semiconductor Quantum Rods , 2001, Science.

[114]  Dimitri Van De Ville,et al.  SUB-RESOLUTION MAXIMUM-LIKELIHOOD BASED LOCALIZATION OF FLUORESCENT NANOPARTICLES IN THREE DIMENSIONS , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[115]  George H Patterson,et al.  Photobleaching and photoactivation: following protein dynamics in living cells. , 2003, Nature cell biology.

[116]  M. J. Cormier,et al.  Primary structure of the Aequorea victoria green-fluorescent protein. , 1992, Gene.

[117]  R. Heintzmann,et al.  Superresolution by localization of quantum dots using blinking statistics. , 2005, Optics express.

[118]  W. Webb,et al.  Precise nanometer localization analysis for individual fluorescent probes. , 2002, Biophysical journal.

[119]  M. Gustafsson,et al.  Subdiffraction Multicolor Imaging of the Nuclear Periphery with 3D Structured Illumination Microscopy , 2008, Science.

[120]  W. Cathey,et al.  Extended depth of field through wave-front coding. , 1995, Applied optics.

[121]  George H. Patterson,et al.  A Photoactivatable GFP for Selective Photolabeling of Proteins and Cells , 2002, Science.

[122]  P. Varga,et al.  Electromagnetic diffraction of light focused through a stratified medium. , 1997, Applied optics.

[123]  Michael J Rust,et al.  Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) , 2006, Nature Methods.

[124]  D A Agard,et al.  Dispersion, aberration and deconvolution in multi‐wavelength fluorescence images , 1996, Journal of microscopy.

[125]  Cedric Nishan Canagarajah,et al.  Image fusion using a new framework for complex wavelet transforms , 2005, IEEE International Conference on Image Processing 2005.

[126]  Dimitri Van De Ville,et al.  An accurate PSF model with few parameters for axially shift-variant deconvolution , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[127]  R. Leitgeb,et al.  Dual-color total internal reflection fluorescence cross-correlation spectroscopy. , 2006, Journal of biomedical optics.

[128]  F. Del Bene,et al.  Optical Sectioning Deep Inside Live Embryos by Selective Plane Illumination Microscopy , 2004, Science.

[129]  Bo Zhang,et al.  Blind deconvolution for diffraction-limited fluorescence microscopy , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[130]  Dimitri Van De Ville,et al.  Model-Based 2.5-D Deconvolution for Extended Depth of Field in Brightfield Microscopy , 2008, IEEE Transactions on Image Processing.