3D endoscope system using DOE projector

For effective in situ endoscopic diagnosis and treatment, size measurement and shape characterization of lesions, such as tumors, is important. For this purpose, in the past we have developed a range of 3D endoscopic systems based on active stereo to measure the shape and size of living tissues. In those works, the main shortcoming was that the target area could only be reconstructed at a specific distance from the scope because of off-focus blurring effects and aberrations in the periphery of the field of view. These issues were compounded by the degree of reconstruction instability due to the strong subsurface scattering common in internal tissue. In this paper, we tackle these shortcomings by developing a new micro pattern laser projector to be inserted in the scope tool channel. The new projector uses a Diffractive Optical Element (DOE) instead of a single lens, which solves the off-focus blur. We also propose a new line-based grid pattern with gap coding to counter the subsurface scattering effect. In our experiments on ex vivo human tumor samples, we show that the tissue shapes were successfully reconstructed regardless of depth variance and strong subsurface scattering effects.

[1]  T. Nagakura,et al.  The study of three-dimensional measurement from an endoscopic images with stereo matching method , 2006, 2006 World Automation Congress.

[2]  Guang-Zhong Yang,et al.  Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[3]  Guang-Zhong Yang,et al.  Metric depth recovery from monocular images using Shape-from-Shading and specularities , 2012, 2012 19th IEEE International Conference on Image Processing.

[4]  Yasushi Yagi,et al.  Grid-Based Active Stereo with Single-Colored Wave Pattern for Dense One-shot 3D Scan , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[5]  Masashi Baba,et al.  2-DOF auto-calibration for a 3D endoscope system based on active stereo , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  K. Deguchi,et al.  Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[7]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[8]  Shinji Tanaka,et al.  Proposal on 3-D endoscope by using grid-based active stereo , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  Shinji Tanaka,et al.  Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  Yasushi Yagi,et al.  Dynamic scene shape reconstruction using a single structured light pattern , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yasushi Yagi,et al.  Dense 3D reconstruction method using a single pattern for fast moving object , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[13]  Branislav Jaramaz,et al.  A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes , 2009, International Journal of Computer Vision.