Deep compressive macroscopic fluorescence lifetime imaging

Compressive Macroscopic Fluorescence Lifetime Imaging (MFLI) is a novel technical implementation that enables monitoring multiple molecular interactions in macroscopic scale. Especially, we reported recently on the development of a hyperspectral wide-field time-resolved single-pixel imaging platform that facilitates whole-body in vivo lifetime imaging in less than 14 minutes. However, despite efficient data acquisition, the data processing of a Compressed Sensing (CS) based inversion plus lifetime fitting remain very time consuming. Herein, we propose to investigate the potential of deep learning for fast and accurate image formation. More precisely we developed a Convolutional Neural Network (CNN) called Net-FLICS (Network for Fluorescence Lifetime Imaging with Compressive Sensing) that reconstructs both intensity and lifetime images directly from raw CS measurements. Results show that better quality reconstruction can be obtained using Net-FLICS, for both simulation and experimental dataset, with almost negligible time compared to the traditional analytic methods. This first investigation suggests that Net-FLICS may be a powerful tool to enable CS-based lifetime imaging for real-time applications.

[1]  Thomas Nowotny,et al.  Artificial neural network approaches for fluorescence lifetime imaging techniques. , 2016, Optics letters.

[2]  Yin Zhang,et al.  An efficient augmented Lagrangian method with applications to total variation minimization , 2013, Computational Optimization and Applications.

[3]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Alix Le Marois,et al.  Fluorescence lifetime imaging (FLIM): Basic concepts and some recent developments , 2015 .

[6]  Xavier Intes,et al.  FLIM-FRET for Cancer Applications. , 2015, Current molecular imaging.

[7]  Xavier Intes,et al.  Time-resolved diffuse optical tomography with patterned-light illumination and detection. , 2010, Optics letters.

[8]  Xavier Intes,et al.  Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging , 2017, Nature Photonics.

[9]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[10]  Xavier Intes,et al.  Assessment of Gate Width Size on Lifetime-Based Förster Resonance Energy Transfer Parameter Estimation , 2015, 2015 41st Annual Northeast Biomedical Engineering Conference (NEBEC).

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Justin Lee,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[14]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements , 2016, ArXiv.

[15]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[18]  Xavier Intes,et al.  Real-time diffuse optical tomography based on structured illumination. , 2010, Journal of biomedical optics.