3D Endoscope System Using Asynchronously Blinking Grid Pattern Projection for HDR Image Synthesis

3D endoscopic systems have been researched and developed to measure the actual shape and size of living tissues for the purpose of remote surgery and diagnosis, to name a few. For such systems, active stereo that consists of a camera and a pattern projector (i.e., structured light systems) is a promising solution because of simple system with high accuracy. Recently, an active-stereo-based 3D endoscope system has been proposed, in which many practical problems were solved such as shallow focal range of the pattern projector or strong diffusion by living tissues. To use the laser pattern projector for endoscopic systems, two fundamental issues arise; a limited dynamic range of the endoscopic camera and a calibration of the system. In this paper, we proposed a new high dynamic range (HDR) image synthesis technique for a laser pattern projector as well as an auto-calibration technique for dynamic motion. Quantitative experiments are conducted to show the effectiveness of the method followed by a demonstration using real endoscopic system.

[1]  Masashi Baba,et al.  2-DOF auto-calibration for a 3D endoscope system based on active stereo , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[2]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[3]  Shinji Tanaka,et al.  Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[4]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[5]  Christophe Doignon,et al.  A structured light-based laparoscope with real-time organs' surface reconstruction for minimally invasive surgery , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  Austin Reiter,et al.  Surgical Structured Light for 3D minimally invasive surgical imaging , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Lena Maier-Hein,et al.  Laparoscopic Quantitative 3D Endoscopy for Image Guided Surgery , 2010, Bildverarbeitung für die Medizin.

[8]  Eli Shechtman,et al.  Patch-based high dynamic range video , 2013, ACM Trans. Graph..

[9]  Ryo Furukawa,et al.  Laser range scanner based on self-calibration techniques using coplanarities and metric constraints , 2009, Comput. Vis. Image Underst..

[10]  Guang-Zhong Yang,et al.  Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery , 2010, MICCAI.

[11]  J. M. M. Montiel,et al.  Visual SLAM for Handheld Monocular Endoscope , 2014, IEEE Transactions on Medical Imaging.

[12]  Shinji Tanaka,et al.  3D endoscope system using DOE projector , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Rafal Mantiuk,et al.  A comparative review of tone‐mapping algorithms for high dynamic range video , 2017, Comput. Graph. Forum.

[14]  Guang-Zhong Yang,et al.  Metric depth recovery from monocular images using Shape-from-Shading and specularities , 2012, 2012 19th IEEE International Conference on Image Processing.

[15]  Lena Maier-Hein,et al.  ToF Meets RGB: Novel Multi-Sensor Super-Resolution for Hybrid 3-D Endoscopy , 2013, MICCAI.

[16]  Yasushi Yagi,et al.  Dense 3D reconstruction method using a single pattern for fast moving object , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  T. Nagakura,et al.  The study of three-dimensional measurement from an endoscopic images with stereo matching method , 2006, 2006 World Automation Congress.

[18]  Ryo Furukawa,et al.  Self-Calibration of Multiple Laser Planes for 3D Scene Reconstruction , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).