High definition images transmission through single multimode fiber using deep learning and simulation speckles

Abstract Multimode fiber (MMF) plays a vital role in promoting the miniaturization of endoscope. However, real-time and high-definition imaging using the MMF that remains a challenging research. Traditional phase compensation and transmission matrix methods are affected by fiber shape and optical devices, which results in low imaging rate and accuracy. Deep learning can be used to construct the inverse transformation matrix (ITM, output to input) of the MMF. However, deep learning requires high similarity between sample sets. In this paper, we combine principal component analysis (PCA) method, deep learning based speckle classification (DLSC) and deep learning based image enhancement (DLIE) to improve imaging definition. To save experimental costs, we use the inverse-PCA method to obtain simulation speckles. The experimental results show that simulation speckles can be used for classification and image reconstruction of experimental speckles. With the difference between simulation and experimental speckles, which brings about low imaging definition. Therefore, we use the DLIE methods to further improve imaging definition. The experimental results show imaging capability with high definition for complex natural scenes, which may provide a feasible method for high definition images transmission through the MMF.

[1]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[2]  Silvio Bianchi,et al.  Hologram transmission through multi-mode optical fibers. , 2011, Optics express.

[3]  Tomáš Čižmár,et al.  Seeing through chaos in multimode fibres , 2015, Nature Photonics.

[4]  Daniele Faccio,et al.  Transmission of natural scene images through a multimode fibre , 2019, Nature Communications.

[5]  Mingying Lan,et al.  Robust compressive multimode fiber imaging against bending with enhanced depth of field. , 2019, Optics express.

[6]  Reza Nasiri Mahalati,et al.  Adaptive control of input field to achieve desired output intensity profile in multimode fiber with random mode coupling. , 2012, Optics express.

[7]  Xiaodong Xu,et al.  A fully stabilized low-phase-noise Kerr-lens mode-locked Yb:CYA laser frequency comb with an average power of 1.5 W , 2020, Applied Physics B.

[8]  Mark C Pierce,et al.  Computational endoscopy-a framework for improving spatial resolution in fiber bundle imaging. , 2019, Optics letters.

[9]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[10]  Jianglei Di,et al.  Deep learning-based object classification through multimode fiber via a CNN-architecture SpeckleNet. , 2018, Applied optics.

[11]  Tom Vercauteren,et al.  Seeing through multimode fibers with real-valued intensity transmission matrices , 2020, Optics express.

[12]  Ioannis N. Papadopoulos,et al.  Focusing and scanning light through a multimode optical fiber using digital phase conjugation. , 2012, Optics express.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  S. Popoff,et al.  Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. , 2009, Physical review letters.

[15]  Guang-Zhong Yang,et al.  Fiber bundle endocytoscopy. , 2013, Biomedical optics express.

[16]  Wonjun Choi,et al.  Transmission matrix of a scattering medium and its applications in biophotonics. , 2015, Optics express.

[17]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[18]  Lei Su,et al.  Deep learning the high variability and randomness inside multimode fibres , 2018, Optics express.

[19]  Jun Tanida,et al.  Object recognition through a multi-mode fiber , 2017 .

[20]  Tomasz S. Tkaczyk,et al.  Achromatized endomicroscope objective for optical biopsy , 2013, Biomedical optics express.

[21]  Tomáš Čižmár,et al.  Shaping the light transmission through a multimode optical fibre: complex transformation analysis and applications in biophotonics. , 2011, Optics express.

[22]  Navid Borhani,et al.  Learning to see through multimode fibers , 2018, Optica.

[23]  K. Dholakia,et al.  Exploiting multimode waveguides for pure fibre-based imaging , 2012, Nature Communications.

[24]  J. John,et al.  Single-mode–multimode–multimode device: sensitivity of the single mode to the fiber parameters and geometrical misalignments , 2016 .

[25]  Haruyoshi Toyoda,et al.  Stable and flexible multiple spot pattern generation using LCOS spatial light modulator. , 2014, Optics express.

[26]  Moonseok Kim,et al.  Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber. , 2012, Physical review letters.

[27]  Tomáš Čižmár,et al.  High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging , 2018, Light: Science & Applications.

[28]  Changhuei Yang,et al.  Relation between speckle decorrelation and optical phase conjugation (OPC)-based turbidity suppression through dynamic scattering media: a study on in vivo mouse skin. , 2015, Biomedical optics express.

[29]  Demetri Psaltis,et al.  Multimode optical fiber transmission with a deep learning network , 2018, Light: Science & Applications.

[30]  Xingzhao Liu,et al.  Imaging through scattering media using speckle pattern classification based support vector regression. , 2018, Optics express.

[31]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Md. Al Mehedi Hasan,et al.  Face recognition using PCA and SVM , 2009, 2009 3rd International Conference on Anti-counterfeiting, Security, and Identification in Communication.

[33]  Navid Borhani,et al.  Imaging through multimode fibers using deep learning: The effects of intensity versus holographic recording of the speckle pattern , 2019, Optical Fiber Technology.

[34]  Perry Ping Shum,et al.  Review of diverse optical fibers used in biomedical research and clinical practice. , 2014, Journal of biomedical optics.

[35]  Jianxin Ma,et al.  Averaging speckle patterns to improve the robustness of compressive multimode fiber imaging against fiber bend. , 2020, Optics express.