Coding Scheme Optimization for Fast Fluorescence Lifetime Imaging

Fluorescence lifetime imaging (FLIM) is used for measuring material properties in a wide range of applications, including biology, medical imaging, chemistry, and material science. In frequency-domain FLIM (FD-FLIM), the object of interest is illuminated with a temporally modulated light source. The fluorescence lifetime is measured by computing the correlations of the emitted light with a demodulation function at the sensor. The signal-to-noise ratio (SNR) and the acquisition time of a FD-FLIM system is determined by the coding scheme (modulation and demodulation functions). In this article, we develop theory and algorithms for designing high-performance FD-FLIM coding schemes that can achieve high SNR and short acquisition time, given a fixed source power budget. Based on a geometric analysis of the image formation and noise model, we propose a novel surrogate objective for the performance of a given coding scheme. The surrogate objective is extremely fast to compute, and can be used to efficiently explore the entire space of coding schemes. Based on this objective, we design novel, high-performance coding schemes that achieve up to an order of magnitude shorter acquisition time as compared to existing approaches. We demonstrate the performance advantage of the proposed schemes in a variety of imaging conditions, using a modular hardware prototype that can implement various coding schemes.

[1]  F. Wouters,et al.  All-solid-state lock-in imaging for wide-field fluorescence lifetime sensing. , 2005, Optics express.

[2]  Takahiro Okabe,et al.  Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Yoichi Sato,et al.  Interreflection Removal Using Fluorescence , 2014, ECCV.

[4]  Wolfgang Heidrich,et al.  Imaging in scattering media using correlation image sensors and sparse convolutional coding. , 2014, Optics express.

[5]  S Schlachter,et al.  mhFLIM: resolution of heterogeneous fluorescence decays in widefield lifetime microscopy. , 2009, Optics express.

[6]  Qionghai Dai,et al.  Decomposing Global Light Transport Using Time of Flight Imaging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Michael W. Davidson,et al.  A bright monomeric green fluorescent protein derived from Branchiostoma lanceolatum , 2013, Nature Methods.

[8]  Jan Siegel,et al.  Time-domain fluorescence lifetime imaging applied to biological tissue , 2004, Photochemical & photobiological sciences : Official journal of the European Photochemistry Association and the European Society for Photobiology.

[9]  Richard A. Keller,et al.  Detection and lifetime measurement of single molecules in flowing sample streams by laser‐induced fluorescence , 1993 .

[10]  Frédo Durand,et al.  Noise-optimal capture for high dynamic range photography , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  K. Carlsson,et al.  Theoretical investigation of the signal-to-noise ratio in fluorescence lifetime imaging. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  P. Seybold,et al.  Solvent Dependence of the Fluorescence Lifetimes of Xanthene Dyes , 1999 .

[13]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[14]  Jens Petter Wold,et al.  Usefulness of fluorescence excitation-emission matrices in combination with PARAFAC, as fingerprints of red wines. , 2009, Journal of agricultural and food chemistry.

[15]  David J. Kriegman,et al.  Shape from Fluorescence , 2012, ECCV.

[16]  Rasmus Bro,et al.  Fluorescence spectroscopy coupled with PARAFAC and PLS DA for characterization and classification of honey. , 2015, Food chemistry.

[17]  E. Gratton,et al.  The phasor approach to fluorescence lifetime imaging analysis. , 2008, Biophysical journal.

[18]  Birgit Ungerböck,et al.  Imaging of oxygen in microreactors and microfluidic systems , 2015, Methods and applications in fluorescence.

[19]  Ramesh Raskar,et al.  Blind and reference-free fluorescence lifetime estimation via consumer time-of-flight sensors , 2015 .

[20]  I. Young,et al.  Photon budget analysis for fluorescence lifetime imaging microscopy. , 2011, Journal of biomedical optics.

[21]  Eleftherios P. Diamandis,et al.  Time-Resolved Fluorescence , 2000 .

[22]  Demetrios Anglos,et al.  Time-resolved fluorescence spectroscopy and imaging of proteinaceous binders used in paintings , 2007, Analytical and bioanalytical chemistry.

[23]  H. Seidel,et al.  Fluorescent immersion range scanning , 2008, ACM Trans. Graph..

[24]  V. M. Murukeshan,et al.  Application of fluorescence lifetime imaging (FLIM) in latent finger mark detection. , 2006, Forensic science international.

[25]  Arthur F Gmitro,et al.  Statistical analysis and optimization of frequency-domain fluorescence lifetime imaging microscopy using homodyne lock-in detection. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[27]  H Szmacinski,et al.  Fluorescence lifetime imaging. , 1992, Analytical biochemistry.

[28]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[29]  J. Lakowicz.  Principles of fluorescence spectroscopy , 1983 .

[30]  M A A Neil,et al.  High-speed wide-field time-gated endoscopic fluorescence-lifetime imaging. , 2004, Optics letters.

[31]  Jacek Kozioł,et al.  Classification of edible oils using synchronous scanning fluorescence spectroscopy , 2005 .

[32]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[33]  Syed Abdullah Aljunid,et al.  Excitation of a single atom with exponentially rising light pulses. , 2013, Physical review letters.

[34]  P. Bastiaens,et al.  Fluorescence lifetime imaging microscopy: spatial resolution of biochemical processes in the cell. , 1999, Trends in cell biology.

[35]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[36]  V. M. Murukeshan,et al.  Fluorescence optimisation and lifetime studies of fingerprints treated with magnetic powders. , 2005, Forensic science international.

[37]  Oscar Beijbom,et al.  Improving Automated Annotation of Benthic Survey Images Using Wide-band Fluorescence , 2016, Scientific Reports.

[38]  Takahiro Okabe,et al.  Bispectral photometric stereo based on fluorescence , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  N. Massari,et al.  SPAD Image Sensor With Analog Counting Pixel for Time-Resolved Fluorescence Detection , 2013, IEEE Transactions on Electron Devices.

[40]  Shree K. Nayar,et al.  What Are Optimal Coding Functions for Time-of-Flight Imaging? , 2018, ACM Trans. Graph..

[41]  J. Hollas.  Modern Spectroscopy , 1936, Nature.

[42]  F. Wouters,et al.  Optimizing frequency-domain fluorescence lifetime sensing for high-throughput applications: photon economy and acquisition speed. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  T Wilson,et al.  Low‐cost, frequency‐domain, fluorescence lifetime confocal microscopy , 2004, Journal of microscopy.

[44]  Moon S. Kim,et al.  Multispectral fluorescence lifetime imaging of feces-contaminated apples by time-resolved laser-induced fluorescence imaging system with tunable excitation wavelengths. , 2008, Applied optics.

[45]  Huimin Ma,et al.  Fluorescent probes and nanoparticles for intracellular sensing of pH values , 2014, Methods and applications in fluorescence.

[46]  S. Achilefu,et al.  Fluorescence lifetime measurements and biological imaging. , 2010, Chemical reviews.

[47]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[48]  M Quarto,et al.  MS‐2 fibrosarcoma characterization by laser induced autofluorescence , 2000, Lasers in surgery and medicine.

[49]  Clemens F Kaminski,et al.  Theoretical investigation of the photon efficiency in frequency-domain fluorescence lifetime imaging microscopy. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  Hans C Gerritsen,et al.  Innovating lifetime microscopy: a compact and simple tool for life sciences, screening, and diagnostics. , 2006, Journal of biomedical optics.

[51]  Ramesh Raskar,et al.  Signal Processing for Time-of-Flight Imaging Sensors: An introduction to inverse problems in computational 3-D imaging , 2016, IEEE Signal Processing Magazine.

[52]  L. Gaboury,et al.  Steady state and time-resolved fluorescence properties of metastatic and non-metastatic malignant cells from different species. , 1995, Journal of photochemistry and photobiology. B, Biology.

[53]  L. Donaldson,et al.  Fluorescence lifetime imaging of lignin autofluorescence in normal and compression wood , 2013, Journal of microscopy.

[54]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[55]  Rinaldo Cubeddu,et al.  Fluorescence lifetime imaging and spectroscopy as tools for nondestructive analysis of works of art. , 2004, Applied optics.

[56]  Hans-Peter Seidel,et al.  Acquisition and analysis of bispectral bidirectional reflectance and reradiation distribution functions , 2010, ACM Trans. Graph..

[57]  Radu Horaud,et al.  Time-of-Flight Cameras , 2012, SpringerBriefs in Computer Science.