Shape Acquisition and Registration for 3D Endoscope Based on Grid Pattern Projection

For effective endoscopic diagnosis and treatment, size measurement and shape characterization of lesions, such as tumors, is important. For this purpose, 3D endoscopic systems based on active stereo to measure the shape and size of living tissue have recently been proposed. In those works, a large problem is the degree of reconstruction instability due to image blurring caused by the strong subsurface scattering common to internal tissue. To reduce this instability problem, using a coarse pattern for structured light is an option, however it reduces the resolution of the acquired shape information. In this paper, we tackle these shortcomings by developing a new micro pattern laser projector to be inserted in the scope tool channel. There are hardware and software contributions in the paper. First, the new projector uses a Diffractive Optical Element (DOE) instead of a single lens which we proposed to solve the off-focus blur. Second, we propose a new line-based grid pattern with gap coding to counter the subsurface scattering effect. The proposed pattern is a coarse grid pattern so that the grid features are not blurred out by the subsurface scattering. Third, to increase shape resolution of line-based grid pattern, we propose to use a multiple shape data registration technique for the grid-structured shapes, which are acquired sequentially by small motions, is proposed. Quantitative experiments are conducted to show the effectiveness of the method followed by a demonstration using real endoscopic system.

[1]  Andrew E. Johnson,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[2]  T. Nagakura,et al.  The study of three-dimensional measurement from an endoscopic images with stereo matching method , 2006, 2006 World Automation Congress.

[3]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[4]  Hongdong Li,et al.  The 3D-3D Registration Problem Revisited , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Masashi Baba,et al.  2-DOF auto-calibration for a 3D endoscope system based on active stereo , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  Atsushi Nakazawa,et al.  Fast simultaneous alignment of multiple range images using index images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[7]  Guang-Zhong Yang,et al.  Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[8]  Jiaolong Yang,et al.  Go-ICP: Solving 3D Registration Efficiently and Globally Optimally , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  K. Deguchi,et al.  Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[10]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[11]  Yasushi Yagi,et al.  Dynamic scene shape reconstruction using a single structured light pattern , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yasushi Yagi,et al.  Dense 3D reconstruction method using a single pattern for fast moving object , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Sing Bing Kang,et al.  Registration and integration of textured 3-D data , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[15]  Shinji Tanaka,et al.  Proposal on 3-D endoscope by using grid-based active stereo , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Joaquim Salvi,et al.  A robust-coded pattern projection for dynamic 3D scene measurement , 1998, Pattern Recognit. Lett..

[17]  Lena Maier-Hein,et al.  ToF Meets RGB: Novel Multi-Sensor Super-Resolution for Hybrid 3-D Endoscopy , 2013, MICCAI.

[18]  Yasushi Yagi,et al.  Grid-Based Active Stereo with Single-Colored Wave Pattern for Dense One-shot 3D Scan , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[19]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[20]  K. Sato,et al.  Range imaging system utilizing nematic liquid crystal mask , 1987 .

[21]  Ryusuke Sagawa,et al.  Dense one-shot 3D reconstruction by detecting continuous regions with parallel line projection , 2011, 2011 International Conference on Computer Vision.

[22]  Taku Komura,et al.  Dynamic Comics for Hierarchical Abstraction of 3D Animation Data , 2013, Comput. Graph. Forum.

[23]  Atsushi Nakazawa,et al.  Parallel alignment of a large number of range images , 2003 .

[24]  Branislav Jaramaz,et al.  A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes , 2009, International Journal of Computer Vision.

[25]  Andrea Tagliasacchi,et al.  Eurographics Symposium on Geometry Processing 2013 Sparse Iterative Closest Point , 2022 .

[26]  Luc Van Gool,et al.  Real-time range acquisition by adaptive structured light , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Guang-Zhong Yang,et al.  Metric depth recovery from monocular images using Shape-from-Shading and specularities , 2012, 2012 19th IEEE International Conference on Image Processing.

[28]  Peter Johannes Neugebauer,et al.  Geometrical cloning of 3D objects via simultaneous registration of multiple range images , 1997, Proceedings of 1997 International Conference on Shape Modeling and Applications.

[29]  Nico Blodow,et al.  Aligning point cloud views using persistent feature histograms , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  W. Kunert,et al.  For 3D laparoscopy: a step toward advanced surgical navigation: how to get maximum benefit from 3D vision , 2013, Surgical Endoscopy.

[31]  Jongmoo Choi,et al.  3D Modeling from Wide Baseline Range Scans Using Contour Coherence , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Takeshi Oishi,et al.  Flying Laser Range Sensor for Large-Scale Site-Modeling and Its Applications in Bayon Digital Archival Project , 2008, International Journal of Computer Vision.

[33]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[34]  Shinji Tanaka,et al.  Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  Sang Wook Lee,et al.  High-Contrast Color-Stripe Pattern for Rapid Structured-Light Range Imaging , 2004, ECCV.

[36]  Lena Maier-Hein,et al.  Laparoscopic Quantitative 3D Endoscopy for Image Guided Surgery , 2010, Bildverarbeitung für die Medizin.

[37]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..