Blind Deconvolution for Confocal Laser Scanning Microscopy

Confocal laser scanning microscopy is a powerful technique for studying biological specimens in three dimensions (3D) by optical sectioning. It permits to visualize images of live specimens non-invasively with a resolution of few hundred nanometers. Although ubiquitous, there are uncertainties in the observation process. As the system's impulse response, or point-spread function (PSF), is dependent on both the specimen and imaging conditions, it should be estimated from the observed images in addition to the specimen. This problem is ill-posed, under-determined. To obtain a solution, it is necessary to insert some knowledge in the form of a priori and adopt a Bayesian approach. The state of the art deconvolution and blind deconvolution algorithms are reviewed within a Bayesian framework. In the first part, we recognize that the diffraction-limited nature of the objective lens and the intrinsic noise are the primary distortions that affect specimen images. An alternative minimization (AM) approach restores the lost frequencies beyond the diffraction limit by using total variation regularization on the object, and a spatial constraint on the PSF. Additionally, some methods are proposed to ensure positivity of estimated intensities, to conserve the object's flux, and to well handle the regularization parameter. When imaging thick specimens, the phase of the pupil function due to spherical aberration (SA) cannot be ignored. It is shown to be dependent on the refractive index mismatch between the object and the objective immersion medium, and the depth under the cover slip. The imaging parameters and the object's original intensity distribution are recovered by modifying the AM algorithm. Due to the incoherent nature of the light in fluorescence microscopy, it is possible to retrieve the phase from the observed intensities by using a model derived from geometrical optics. This was verified on the simulated data. This method could also be extended to restore specimens affected by SA. As the PSF is space varying, a piecewise convolution model is proposed, and the PSF approximated so that, apart from the specimen, it is sufficient to estimated only one free parameter.

[1]  H. H. Hopkins The frequency response of a defocused optical system , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[2]  J. Goodman Introduction to Fourier optics , 1969 .

[3]  P. Stokseth Properties of a Defocused Optical System , 1969 .

[4]  K. Miller Least Squares Methods for Ill-Posed Problems with a Prescribed Bound , 1970 .

[5]  R. Gerchberg A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[6]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[7]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[8]  E. Anderson Hudson et al. , 1977 .

[9]  H. Walker Quasi-Newton Methods , 1978 .

[10]  L. Mandel,et al.  Sub-Poissonian photon statistics in resonance fluorescence. , 1979, Optics letters.

[11]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[12]  Michael A. Fiddy,et al.  The essential role of prior knowledge in phase retrieval , 1984 .

[13]  D. Agard Optical sectioning microscopy: cellular architecture in three dimensions. , 1984, Annual review of biophysics and bioengineering.

[14]  J Bille,et al.  Reconstructing 3-D light-microscopic images by digital image processing. , 1985, Applied optics.

[15]  Edward S. Meinel,et al.  Origins of linear and nonlinear recursive restoration algorithms , 1986 .

[16]  C. J. R. Sheppard,et al.  The spatial frequency cut-off in three-dimensional imaging II , 1986 .

[17]  R. Howard,et al.  Local convergence analysis of a grouped variable version of coordinate descent , 1987 .

[18]  J. C. Dainty,et al.  Iterative blind deconvolution method and its applications , 1988 .

[19]  Timothy J. Holmes Maximum-likelihood image restoration adapted for noncoherent optical imaging , 1988 .

[20]  C. Sheppard,et al.  Depth of field in optical microscopy , 1988 .

[21]  M. Minsky Memoir on inventing the confocal scanning microscope , 1988 .

[22]  Guy Demoment,et al.  Image reconstruction and restoration: overview of common estimation structures and problems , 1989, IEEE Trans. Acoust. Speech Signal Process..

[23]  S. Gibson,et al.  Diffraction by a circular aperture as a model for three-dimensional optical microscopy. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[24]  David A. Agard,et al.  Solution of Toeplitz systems for the restoration of 3-D optical-sectioning microscopy data , 1990, Photonics West - Lasers and Applications in Science and Engineering.

[25]  F. S. Fay,et al.  3D Fluorescence imaging of single cells using image restoration , 1990 .

[26]  J. Pawley,et al.  Handbook of Biological Confocal Microscopy , 1990, Springer US.

[27]  I. Csiszár Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems , 1991 .

[28]  L. J. Thomas,et al.  Regularized linear method for reconstruction of three-dimensional microscopic objects from optical sections. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[29]  T J Holmes,et al.  Blind deconvolution of quantum-limited incoherent imagery: maximum-likelihood approach. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[30]  T. J. Keating,et al.  Thin-section ratiometric Ca2+ images obtained by optical sectioning of fura-2 loaded mast cells , 1992, The Journal of cell biology.

[31]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[32]  G. J. Brakenhoff,et al.  Refractive index and axial distance measurements in 3-D microscopy , 1992 .

[33]  Michel Barlaud,et al.  An adaptive reconstruction method involving discontinuities , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[34]  S. Joshi,et al.  Maximum a posteriori estimation with Good's roughness for three-dimensional optical-sectioning microscopy. , 1993, Journal of the Optical Society of America. A, Optics and image science.

[35]  Alberto Diaspro,et al.  3-D reconstruction in optical microscopy by a frequency-domain approach , 1993, Signal Process..

[36]  Jean-Luc Starck,et al.  Filtering and deconvolution by the wavelet transform , 1994, Signal Process..

[37]  L. J. Thomas,et al.  Artifacts in computational optical-sectioning microscopy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[38]  Judy E. Trogadis,et al.  Three-dimensional confocal microscopy : volume investigation of biological specimens , 1994 .

[39]  S. Hell,et al.  Lens Aberrations in Confocal Fluorescence Microscopy , 1995 .

[40]  Peter Török,et al.  Electromagnetic diffraction of light focused through a planar interface between materials of mismatched refractive indices: an integral representation , 1995 .

[41]  Deepa Kundur,et al.  Blind image restoration via recursive filtering using deterministic constraints , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[42]  A. Mohammad-Djafari A full Bayesian approach for inverse problems , 2001, physics/0111123.

[43]  Chin-Tu Chen,et al.  Optical microscopy system for 3D dynamic imaging , 1996, Electronic Imaging.

[44]  G. M. P. VAN KEMPEN,et al.  A quantitative comparison of image restoration methods for confocal microscopy , 1997 .

[45]  D S Biggs,et al.  Acceleration of iterative image restoration algorithms. , 1997, Applied optics.

[46]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[47]  David A. Agard,et al.  Three‐dimensional microscopy in thick biological samples: A fresh approach for adjusting focus and correcting spherical aberration , 1997 .

[48]  Peter Varga,et al.  The role of specimen‐induced spherical aberration in confocal microscopy , 1997 .

[49]  Peter J. Verveer,et al.  Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy , 1997 .

[50]  T. Wilson,et al.  Aberration correction for confocal imaging in refractive‐index‐mismatched media , 1998 .

[51]  Deepa Kundur,et al.  A novel blind deconvolution scheme for image restoration using recursive filtering , 1998, IEEE Trans. Signal Process..

[52]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[53]  Teh-hui Kao,et al.  A GFP–MAP4 Reporter Gene for Visualizing Cortical Microtubule Rearrangements in Living Epidermal Cells , 1998, Plant Cell.

[54]  S. Hell,et al.  Refractive index mismatch induced intensity and phase variations in fluorescence confocal, multiphoton and 4Pi-microscopy. , 1998 .

[55]  J. Lindsey,et al.  PhotochemCAD ‡ : A Computer‐Aided Design and Research Tool in Photochemistry , 1998 .

[56]  Yoram Bresler,et al.  Perfect blind restoration of images blurred by multiple filters: theory and efficient algorithms , 1999, IEEE Trans. Image Process..

[57]  A. Carasso Linear and Nonlinear Image Deblurring: A Documented Study , 1999 .

[58]  J. Conchello,et al.  Parametric blind deconvolution: a robust method for the simultaneous estimation of image and blur. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[59]  P. Verveer,et al.  A comparison of image restoration approaches applied to three‐dimensional confocal and wide‐field fluorescence microscopy , 1999, Journal of microscopy.

[60]  I T Young,et al.  Model-based resolution: applying the theory in quantitative microscopy. , 2000, Applied optics.

[61]  Geert M. P. van Kempen,et al.  Background estimation in nonlinear image restoration , 2000 .

[62]  E. Bratsolis,et al.  A spatial regularization method preserving local photometry for Richardson-Lucy restoration , 2001 .

[63]  David A. Agard,et al.  Phase retrieval of widefield microscopy point spread functions , 2001, SPIE BiOS.

[64]  D. Agard,et al.  Computational adaptive optics for live three-dimensional biological imaging , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[65]  M. Persson,et al.  Total variation norm for three-dimensional iterative reconstruction in limited view angle tomography , 2001, Physics in medicine and biology.

[66]  Robert D. Nowak,et al.  Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative prior , 2001, IEEE Trans. Image Process..

[67]  Mário A. T. Figueiredo,et al.  Wavelet-Based Image Estimation : An Empirical Bayes Approach Using Jeffreys ’ Noninformative Prior , 2001 .

[68]  Alfred S. Carasso,et al.  Direct Blind Deconvolution , 2001, SIAM J. Appl. Math..

[69]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[70]  J Boutet de Monvel,et al.  Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ. , 2001, Biophysical journal.

[71]  O. Albert,et al.  Adaptive correction of depth‐induced aberrations in multiphoton scanning microscopy using a deformable mirror , 2002, Journal of microscopy.

[72]  A. Carasso THE APEX METHOD IN IMAGE SHARPENING AND THE USE OF LOW EXPONENT LÉVY STABLE LAWS , 2002 .

[73]  T. Wilson,et al.  Adaptive aberration correction in a confocal microscope , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[74]  Josiane Zerubia,et al.  Hyperparameter estimation for satellite image restoration using a MCMC maximum-likelihood method , 2002, Pattern Recognit..

[75]  John W Sedat,et al.  Phase retrieval for high-numerical-aperture optical systems. , 2003, Optics letters.

[76]  M. Jiang,et al.  Development of blind image deconvolution and its applications. , 2003, Journal of X-ray science and technology.

[77]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[78]  Alfred S. Carasso The APEX Method in Image Sharpening and the Use of Low Exponent L[e-acute]vy Stable Laws , 2003, SIAM J. Appl. Math..

[79]  Josiane Zerubia,et al.  Wavelet-based superresolution in astronomy. , 2003 .

[80]  Josiane Zerubia,et al.  Satellite Image Deblurring Using Complex Wavelet Packets , 2003, International Journal of Computer Vision.

[81]  Alain Dieterlen,et al.  Identification and restoration in 3D fluorescence microscopy , 2004, International Conference on Correlation Optics.

[82]  Chrysanthe Preza,et al.  Depth-variant maximum-likelihood restoration for three-dimensional fluorescence microscopy. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[83]  J. Zerubia,et al.  3D Microscopy Deconvolution using Richardson-Lucy Algorithm with Total Variation Regularization , 2004 .

[84]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[85]  Charles Kervrann,et al.  An adaptive window approach for Poisson noise reduction and structure preserving in confocal microscopy , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[86]  Zhiping Lin,et al.  Noise suppression of point spread functions and its influence on deconvolution of three‐dimensional fluorescence microscopy image sets , 2005, Journal of microscopy.

[87]  S. Alenius,et al.  Bayesian image reconstruction for emission tomography based on median root prior , 1997, European Journal of Nuclear Medicine.

[88]  J. Sibarita Deconvolution microscopy. , 2005, Advances in biochemical engineering/biotechnology.

[89]  James G. Nagy,et al.  BLIND ITERATIVE RESTORATION OF IMAGES WITH SPATIALLY-VARYING BLUR , 2005 .

[90]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[91]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[92]  R. Juškaitis Measuring the Real Point Spread Function of High Numerical Aperture Microscope Objective Lenses , 2006 .

[93]  Edgar Brunner,et al.  In situ background estimation in quantitative fluorescence imaging. , 2006, Biophysical journal.

[94]  Mark B. Cannell,et al.  Image Enhancement by Deconvolution , 2006 .

[95]  Josiane Zerubia,et al.  Richardson–Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution , 2006, Microscopy research and technique.

[96]  Stuart Jefferies,et al.  A computational method for the restoration of images with an unknown, spatially-varying blur. , 2006, Optics express.

[97]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[98]  A. Nehorai,et al.  Deconvolution methods for 3-D fluorescence microscopy images , 2006, IEEE Signal Processing Magazine.

[99]  S. Inoué,et al.  Foundations of Confocal Scanned Imaging in Light Microscopy , 2006 .

[100]  Laurent M. Mugnier,et al.  3D deconvolution of adaptive-optics corrected retinal images , 2006, SPIE BiOS.

[101]  S. Shaw Imaging the live plant cell. , 2006, The Plant journal : for cell and molecular biology.

[102]  James B. Pawley,et al.  Fundamental Limits in Confocal Microscopy , 2006 .

[103]  Joseph Schlecht,et al.  Statistical Inference of Biological Structure and Point Spread Functions in 3D Microscopy , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[104]  G. Aubert,et al.  Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations (Applied Mathematical Sciences) , 2006 .

[105]  J. Zerubia,et al.  Gaussian approximations of fluorescence microscope point-spread function models. , 2007, Applied optics.

[106]  Rafael Molina,et al.  Blind Image Deconvolution: Problem formulation and existing approaches , 2007 .

[107]  Josiane Zerubia,et al.  Bayesian estimation of blur and noise in remote sensing imaging , 2007 .

[108]  J. Zerubia,et al.  Parametric Blind Deconvolution for Confocal Laser Scanning Microscopy , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[109]  Franck Marchis,et al.  AIDA: an adaptive image deconvolution algorithm with application to multi-frame and three-dimensional data. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[110]  Satyandra K. Gupta,et al.  Algorithms for On-Line Monitoring of Micro Spheres in an Optical Tweezers-Based Assembly Cell , 2007, J. Comput. Inf. Sci. Eng..

[111]  John W Sedat,et al.  Modelling the application of adaptive optics to wide‐field microscope live imaging , 2007, Journal of microscopy.

[112]  Oleg V. Michailovich,et al.  Deconvolution of Medical Images from Microscopic to Whole Body Images , 2007 .

[113]  Joshua W Shaevitz,et al.  Enhanced three-dimensional deconvolution microscopy using a measured depth-varying point-spread function. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[114]  Bo Zhang,et al.  Contributions à la microscopie à fluorescence en imagerie biologique : modélisation de la PSF, restauration d'images et détection super-résolutive. (Contributions to fluorescence microscopy in biological imaging: PSF modeling, image restoration, and super-resolution detection) , 2007 .

[115]  K. Egiazarian,et al.  Blind image deconvolution , 2007 .

[116]  Josiane Zerubia,et al.  Wavelet-based restoration methods: application to 3D confocal microscopy images , 2007, SPIE Optical Engineering + Applications.

[117]  M. Booth Adaptive optics in microscopy. , 2003, Philosophical transactions. Series A, Mathematical, physical, and engineering sciences.

[118]  Michael Unser,et al.  FAST WAVELET-REGULARIZED IMAGE DECONVOLUTION , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[119]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[120]  Jean-Luc Starck,et al.  Deconvolution and Blind Deconvolution in Astronomy , 2007 .

[121]  Mohamed-Jalal Fadili,et al.  Image deconvolution under poisson noise using sparse representations and proximal thresholding iteration , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[122]  Bo Zhang,et al.  Improving single particle localization with an empirically calibrated Gaussian kernel , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[123]  Bruno Colicchio,et al.  Fast deconvolution with non-invariant PSF for 3-D fluorescence microscopy , 2008, SPIE Photonics Europe.

[124]  Pierre Weiss,et al.  Algorithmes rapides d'optimisation convexe. Applications à la reconstruction d'images et à la détection de changements. (Fast algorithms for convex optimization. Applications to image reconstruction and change detection) , 2008 .

[125]  O. Haeberlé,et al.  Tomographic diffractive microscopy of transparent samples , 2008 .

[126]  Bo Zhang,et al.  Blind deconvolution for diffraction-limited fluorescence microscopy , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[127]  Martin Burger,et al.  Bregman-EM-TV Methods with Application to Optical Nanoscopy , 2009, SSVM.

[128]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[129]  Josiane Zerubia,et al.  Blind deconvolution for thin-layered confocal imaging. , 2009, Applied optics.

[130]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[131]  Brendt Wohlberg,et al.  Efficient Minimization Method for a Generalized Total Variation Functional , 2009, IEEE Transactions on Image Processing.

[132]  Josiane Zerubia,et al.  Point-Spread Function retrieval for fluorescence microscopy , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[133]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[134]  Giuseppe Vicidomini,et al.  Automatic deconvolution of 4Pi-microscopy data with arbitrary phase. , 2009, Optics letters.

[135]  Josiane Zerubia,et al.  Complex wavelet regularization for solving inverse problems in remote sensing , 2009, 2009 IEEE International Geoscience and Remote Sensing Symposium.

[136]  Animesh Khemka Inverse problems in image processing , 2009 .

[137]  Space non-invariant point-spread function and its estimation in fluorescence microscopy , 2009 .

[138]  Thierry Blu,et al.  Fast interscale wavelet denoising of Poisson-corrupted images , 2010, Signal Process..

[139]  D. Malacara-Hernández,et al.  PRINCIPLES OF OPTICS , 2011 .