Computational and optical methods for improving resolution and signal quality in fluorescence microscopy

We present efficient algorithms for image restoration using the maximum a posteriori (MAP) method. Assuming Gaussian or Poisson statistics of the noise and either a Gaussian or an entropy prior distribution for the image, corresponding functionals are formulated and minimized to produce MAP estimations. Efficient algorithms are presented for finding the minimum of these functionals in the presence of non-negativity and support constraints. Performance was tested by using simulated three-dimensional (3D) imaging with a fluorescence confocal laser scanning microscope. Results are compared with those from two existing algorithms for superresolution in fluorescence imaging. An example is given of the restoration of a 3D confocal image of a biological specimen.

[1]  Rishi Kant,et al.  An analytical solution of vector diffraction for focusing optical systems , 1993 .

[2]  T M Jovin,et al.  Fluorescence labeling and microscopy of DNA. , 1989, Methods in cell biology.

[3]  李幼升,et al.  Ph , 1989 .

[4]  B. Frieden,et al.  Optical Transfer of the Three-Dimensional Object*† , 1967 .

[5]  Adriaan van den Bos,et al.  Resolution: a survey , 1997 .

[6]  Peter J. Verveer,et al.  Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy , 1997 .

[7]  J. Skilling,et al.  Probabilistic data analysis: an introductory guide , 1998 .

[8]  Thomas M. Jovin,et al.  Super-resolution MAP algorithms applied to fluorescence imaging , 1997, Photonics West - Biomedical Optics.

[9]  W. Campbell,et al.  Raman microspectroscopy of intracellular cholesterol crystals in cultured bovine coronary artery endothelial cells. , 1997, Journal of lipid research.

[10]  Aggelos K. Katsaggelos,et al.  A regularized iterative image restoration algorithm , 1991, IEEE Trans. Signal Process..

[11]  Bobby R. Hunt,et al.  Improved Methods of Maximum a Posteriori Restoration , 1979, IEEE Transactions on Computers.

[12]  H T van der Voort,et al.  Three-dimensional confocal fluorescence microscopy. , 1989, Methods in cell biology.

[13]  K. Miller.  Least Squares Methods for Ill-Posed Problems with a Prescribed Bound , 1970 .

[14]  Jose-Angel Conchello,et al.  Tradeoffs in regularized maximum-likelihood image restoration , 1997, Photonics West - Biomedical Optics.

[15]  M. Manfait,et al.  Characterization of Acidic Vesicles in Multidrug-resistant and Sensitive Cancer Cells by Acridine Orange Staining and Confocal Microspectrofluorometry , 1997, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[16]  Nikolas P. Galatsanos,et al.  Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation , 1992, IEEE Trans. Image Process..

[17]  David A. Agard,et al.  Three-dimensional architecture of a polytene nucleus , 1983, Nature.

[18]  Joseph A. O'Sullivan,et al.  Deblurring subject to nonnegativity constraints , 1992, IEEE Trans. Signal Process..

[19]  T M Jovin,et al.  Imaging the intracellular trafficking and state of the AB5 quaternary structure of cholera toxin. , 1996, The EMBO journal.

[20]  N. Streibl.  Three-dimensional imaging by a microscope , 1985 .

[21]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[22]  J. L. Harris,et al.  Diffraction and Resolving Power , 1964 .

[23]  Kristofer S. J. Pister,et al.  MICRO-OPTO-ELECTRO-MECHANICAL DEVICES AND ON-CHIP OPTICAL PROCESSING , 1997 .

[24]  L. Landweber.  An iteration formula for Fredholm integral equations of the first kind , 1951 .

[25]  J. Greve,et al.  Studying single living cells and chromosomes by confocal Raman microspectroscopy , 1990, Nature.

[26]  M. Kozubek,et al.  Efficient real-time confocal microscopy with white light sources , 1996, Nature.

[27]  Charles J. Koester,et al.  Comparison of Various Optical Sectioning Methods , 1995 .

[28]  David M. Haaland,et al.  Multivariate Classification of the Infrared Spectra of Cell and Tissue Samples , 1997 .

[29]  Mark D. Fricker,et al.  Aberration control in quantitative imaging of botanical specimens by multidimensional fluorescence microscopy , 1996 .

[30]  D. Arndt-Jovin,et al.  FISH in whole‐mount Drosophila embryos. RNA: activation of a transcriptional locus, DNA: gene architecture and expression , 1996 .

[31]  I. Csiszár.  Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[32]  Colin J. R. Sheppard,et al.  Signal-to-Noise in Confocal Microscopes , 1995 .

[33]  R. N. Hook,et al.  HST image restoration developments at the ST‐ECF , 1995, Int. J. Imaging Syst. Technol..

[34]  Peter Török,et al.  Electromagnetic diffraction of light focused through a planar interface between materials of mismatched refractive indices: structure of the electromagnetic field. I , 1995 .

[35]  M. D. Egger,et al.  New Reflected-Light Microscope for Viewing Unstained Brain and Ganglion Cells , 1967, Science.

[36]  T M Jovin,et al.  Luminescence digital imaging microscopy. , 1989, Annual review of biophysics and biophysical chemistry.

[37]  L. J. van Vliet,et al.  Grey-Scale Measurements in Multi-Dimensional Digitized Images , 1993 .

[38]  J. Conchello,et al.  Enhanced 3-D reconstruction from confocal scanning microscope images. 1: Deterministic and maximum likelihood reconstructions. , 1990, Applied optics.

[39]  Thomas M. Jovin,et al.  Acceleration of the ICTM image restoration algorithm , 1997 .

[40]  Mario Bertero,et al.  Regularization methods in image restoration: An application to HST images , 1995, Int. J. Imaging Syst. Technol..

[41]  Peter Varga,et al.  Analytical solution of the diffraction integrals and interpretation of wave-front distortion when light is focused through a planar interface between materials of mismatched refractive indices , 1995 .

[42]  G. B. Avinash,et al.  Data‐driven, simultaneous blur and image restoration in 3‐D fluorescence microscopy , 1996 .

[43]  M. Kozubek,et al.  Confocal microscopy by aperture correlation. , 1996, Optics letters.

[44]  Stephen F. Gull,et al.  Developments in Maximum Entropy Data Analysis , 1989 .

[45]  P. Stokseth.  Properties of a Defocused Optical System , 1969 .

[46]  N. Streibl,et al.  Fundamental restrictions for 3-D light distributions , 1984 .

[47]  T. Holmes,et al.  Acceleration of Maximum-Likelihood Image-Restoration for Fluorescence Microscopy and Other Noncoherent Imagery , 1991, Quantum Limited Imaging and Information Processing.

[48]  Colin J. R. Sheppard,et al.  The significance of 3‐D transfer functions in confocal scanning microscopy , 1992 .

[49]  B. R. Hunt.  Prospects for Image Restoration , 1994 .

[50]  T J Holmes,et al.  Richardson-Lucy/maximum likelihood image restoration algorithm for fluorescence microscopy: further testing. , 1989, Applied optics.

[51]  G. J. Brakenhoff,et al.  Refractive index and axial distance measurements in 3-D microscopy , 1992 .

[52]  P. H. Cittert.  Zum Einfluß der Spaltbreite auf die Intensitätsverteilung in Spektrallinien. II , 1930 .

[53]  D. Ben‐Amotz,et al.  Description and Theory of a Fiber-Optic Confocal and Super-Focal Raman Microspectrometer , 1996 .

[54]  S. Joshi,et al.  Maximum a posteriori estimation with Good's roughness for three-dimensional optical-sectioning microscopy. , 1993, Journal of the Optical Society of America. A, Optics and image science.

[55]  Direct View Confocal Imaging Systems Using a Slit Aperture , 1995 .

[56]  S. Gibson,et al.  Diffraction by a circular aperture as a model for three-dimensional optical microscopy. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[57]  G. J. Brakenhoff,et al.  Determination of the 3-dimensional optical properties of a confocal scanning laser microscope , 1988 .

[58]  Spyridon S. Stefanou,et al.  Maximum-likelihood reconstruction of 3D confocal data sets , 1994, Electronic Imaging.

[59]  W. G. Fateley,et al.  Chemical Mapping in the Mid- and Near-IR Spectral Regions by Hadamard Transform/FT-IR Spectrometry , 1997 .

[60]  Peter Török,et al.  Electromagnetic diffraction of light focused through a planar interface between materials of mismatched refractive indices: an integral representation , 1995 .

[61]  S. J. Hewlett,et al.  Optical sectioning strength of the direct‐view microscope employing finite‐sized pin‐hole arrays , 1991 .

[62]  P. Treado,et al.  Multichannel Hadamard Transform Raman Microscopy , 1990 .

[63]  Peter J. Verveer,et al.  Optical Sectioning Fluorescence Spectroscopy in a Programmable Array Microscope , 1998 .

[64]  David A. Agard,et al.  Three‐dimensional microscopy in thick biological samples: A fresh approach for adjusting focus and correcting spherical aberration , 1997 .

[65]  J. Pawley,et al.  Handbook of Biological Confocal Microscopy , 1990, Springer US.

[66]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[67]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[68]  T. Bisseling,et al.  Microspectroscopic imaging of nodulation factor-binding sites on living Vicia sativa roots using a novel bioactive fluorescent nodulation factor. , 1997, Biophysical journal.

[69]  R. A. Gaskins,et al.  Nonparametric roughness penalties for probability densities , 2022 .

[70]  R. White,et al.  Image recovery from data acquired with a charge-coupled-device camera. , 1993, Journal of the Optical Society of America. A, Optics and image science.

[71]  H. H. Hopkins.  The frequency response of a defocused optical system , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[72]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[73]  Eithne M. McCabe,et al.  Direct‐view microscopy: experimental investigation of the dependence of the optical sectioning characteristics on pinhole‐array configuration , 1997 .

[74]  Bobby R. Hunt,et al.  Super‐resolution of images: Algorithms, principles, performance , 1995, Int. J. Imaging Syst. Technol..

[75]  Geert M. P. van Kempen,et al.  Comparing maximum likelihood estimation and constrained Tikhonov-Miller restoration , 1996 .

[76]  Jack Yarwood,et al.  Depth Profiling of Poly(Methyl Methacrylate), Poly(Vinyl Alcohol) Laminates by Confocal Raman Microspectroscopy , 1996 .

[77]  V. Evangelista,et al.  Confocal‐line microscopy , 1992, Journal of microscopy.

[78]  Tuan Vo-Dinh,et al.  Laser-Induced Differential Fluorescence for Cancer Diagnosis without Biopsy , 1997 .

[79]  Chusuke Munakata,et al.  Calculation of three-dimensional optical transfer function for a confocal scanning fluorescent microscope , 1989 .

[80]  S W Hell,et al.  Far‐field fluorescence microscopy with three‐dimensional resolution in the 100‐nm range , 1997, Journal of microscopy.

[81]  R. Mersereau,et al.  Iterative methods for image deblurring , 1990 .

[82]  Søren I. Olsen,et al.  Estimation of Noise in Images: An Evaluation , 1993, CVGIP Graph. Model. Image Process..

[83]  Chusuke Munakata,et al.  Dependence of 3-D optical transfer functions on the pinhole radius in a fluorescent confocal optical microscope. , 1990, Applied optics.

[84]  W. Denk,et al.  Two-photon laser scanning fluorescence microscopy. , 1990, Science.

[85]  Larry J. Hornbeck,et al.  Deformable-Mirror Spatial Light Modulators , 1990, Optics & Photonics.

[86]  L. J. Thomas,et al.  Artifacts in computational optical-sectioning microscopy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[87]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[88]  Stefan W. Hell,et al.  A confocal beam scanning white‐light microscope , 1991 .

[89]  L. Lucy.  An iterative technique for the rectification of observed distributions , 1974 .

[90]  J. Lichtman,et al.  Theoretical analysis of a rotating-disk partially confocal scanning microscope. , 1994, Applied optics.

[91]  H. M. Voort,et al.  Restoration of confocal images for quantitative image analysis , 1995 .

[92]  C. Sheppard,et al.  Effects of specimen refractive index on confocal imaging , 1997 .

[93]  D. Agard,et al.  Fluorescence microscopy in three dimensions. , 1989, Methods in cell biology.

[94]  Badrinath Roysam,et al.  Light Microscopic Images Reconstructed by Maximum Likelihood Deconvolution , 1995 .

[95]  Peter J. Verveer,et al.  Image restoration based on Good’s roughness penalty with application to fluorescence microscopy , 1998 .

[96]  C. Sheppard,et al.  Confocal fluorescent microscopy with a finite-sized circular detector , 1992 .

[97]  Aggelos K. Katsaggelos,et al.  Iterative identification and restoration of images , 1990, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[98]  T. Wilson,et al.  Optical sectioning in confocal fluorescent microscopes , 1989 .

[99]  M. Delhaye,et al.  Raman confocal microprobing, imaging and fibre‐optic remote sensing: A further step in molecular analysis , 1994 .

[100]  Marcel J. E. Golay,et al.  Complementary series , 1961, IRE Trans. Inf. Theory.

[101]  Colin J. R. Sheppard,et al.  Axial resolution of confocal fluorescence microscopy , 1989 .

[102]  D. Agard,et al.  Determination of three-dimensional imaging properties of a light microscope system. Partial confocal behavior in epifluorescence microscopy. , 1990, Biophysical journal.

[103]  Eric W. Hansen,et al.  Three-Dimensional Reconstruction Of Noisy Confocal Scanning Microscope Images , 1989, Optics & Photonics.

[104]  B. Hunt,et al.  Analysis of the limit to superresolution in incoherent imaging , 1993 .

[105]  J N Turner,et al.  Blind deconvolution of fluorescence micrographs by maximum-likelihood estimation. , 1995, Applied optics.

[106]  G. Kino.  Intermediate Optics in Nipkow Disk Microscopes , 1990 .

[107]  D. Agard.  Optical sectioning microscopy: cellular architecture in three dimensions. , 1984, Annual review of biophysics and bioengineering.

[108]  W. A. Carrington.  Image restoration in 3-D microscopy with limited data , 1990, Photonics West - Lasers and Applications in Science and Engineering.

[109]  C. J. R. Sheppard,et al.  The spatial frequency cut-off in three-dimensional imaging II , 1986 .

[110]  S. Hell,et al.  Aberrations in confocal fluorescence microscopy induced by mismatches in refractive index , 1993 .

[111]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[112]  Luo Zhengrong,et al.  Study of Diagnosis Criteria for Pathological Tissues by Laser-Induced Fluorescence , 1997 .

[113]  T J Holmes,et al.  Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[114]  Tony Wilson,et al.  Three dimensional image reconstruction in conventional and confocal microscopy , 1993 .

[115]  J. Skilling.  Classic Maximum Entropy , 1989 .

[116]  R. Mersereau,et al.  Optimal estimation of the regularization parameter and stabilizing functional for regularized image restoration , 1990 .

[117]  G. M. P. VAN KEMPEN,et al.  A quantitative comparison of image restoration methods for confocal microscopy , 1997 .

[118]  William G. Fateley,et al.  A Hadamard-multiplexed spectrometer based on an acousto-optic tunable filter , 1996 .

[119]  Aggelos K. Katsaggelos,et al.  New termination rule for linear iterative image restoration algorithms , 1990 .

[120]  P. Shaw,et al.  Comparison of Wide-Field/Deconvolution and Confocal Microscopy for 3D Imaging , 1995 .

[121]  G. Toraldo di Francia.  Degrees of freedom of an image. , 1969, Journal of the Optical Society of America.

[122]  Enzo Benedetti,et al.  Determination of the Relative Amount of Nucleic Acids and Proteins in Leukemic and Normal Lymphocytes by Means of Fourier Transform Infrared Microspectroscopy , 1997 .

[123]  M. Bertero.  Linear Inverse and III-Posed Problems , 1989 .

[124]  Colin J. R. Sheppard,et al.  Approximation to the three-dimensional optical transfer function , 1991 .

[125]  M I Miller,et al.  Bayesian image reconstruction for emission tomography incorporating Good's roughness prior on massively parallel processors. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[126]  Geert M. P. van Kempen,et al.  Avalanche photodiode detection with object scanning and image restoration provides 2–4 fold resolution increase in two‐photon fluorescence microscopy , 1996 .

[127]  T. Holmes.  Expectation-maximization restoration of band-limited, truncated point-process intensities with application in microscopy , 1989 .

[128]  Timothy J. Holmes.  Maximum-likelihood image restoration adapted for noncoherent optical imaging , 1988 .

[129]  Valtere Evangelista,et al.  Achieving confocal‐point performance in confocal‐line microscopy , 1994 .

[130]  Colin J. R. Sheppard,et al.  An electromagnetic theory of imaging in fluorescence microscopy, and imaging in polarization fluorescence microscopy , 1997 .

[131]  W. Webb,et al.  Background rejection and signal-to-noise optimization in confocal and alternative fluorescence microscopes. , 1994, Applied optics.

[132]  F S Fay,et al.  Superresolution three-dimensional images of fluorescence in cells with minimal light exposure. , 1995, Science.

[133]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[134]  T. Jovin,et al.  Theory of confocal fluorescence imaging in the programmable array microscope (PAM) , 1998 .

[135]  G. J. Brakenhoff,et al.  Real-Time Stereo (3D) Confocal Microscopy , 1995 .

[136]  Martin Schrader,et al.  Potential of confocal microscopes to resolve in the 50–100 nm range , 1996 .

[137]  D. M. Titterington,et al.  A Study of Methods of Choosing the Smoothing Parameter in Image Restoration by Regularization , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[138]  N. Sloane,et al.  Hadamard transform optics , 1979 .

[139]  M. Morris,et al.  Optical Sectioning Raman Microscopy , 1991 .

[140]  Taco D. Visser,et al.  ELECTROMAGNETIC DESCRIPTION OF IMAGE FORMATION IN CONFOCAL FLUORESCENCE MICROSCOPY , 1994 .

[141]  M. Liang,et al.  Confocal pattern period in multiple-aperture confocal imaging systems with coherent illumination. , 1997, Optics letters.

[142]  Patrick J. Treado,et al.  Imaging Spectrometers for Fluorescence and Raman Microscopy: Acousto-Optic and Liquid Crystal Tunable Filters , 1994 .

[143]  G. D. Francia.  Degrees of Freedom of Image , 1969 .

[144]  F. Groen,et al.  The one-point fluorescence response in confocal microscopy , 1991 .