Surgical Structured Light for 3D minimally invasive surgical imaging

Surgeons perform minimally invasive surgery using an image delivered by a laparoscope and a camera system that provides a high definition 2D image, but this leaves the surgeon without 3D depth perception. The lack of depth perception can slow the surgeon, increase the risk of misidentifying structures, and/or inadvertently cause unwanted injury to tissues surrounding the surgical site. To address the lack of depth perception, we propose a Surgical Structured Light (SSL) system that includes a 3D sensor capable of measuring and modeling the surgical site during a procedure. The 3D information provided by this system can enable the surgeon to: 1) improve the navigation of tools based on precise localization of instruments in relation to structures in the surgical site, 2) allow 3D visualizations side-by-side with a standard 2D color image, and 3) precisely measure sizes of structures (e.g., tumors) and distances between structures with simple mouse clicks. We demonstrate the accuracy of our SSL system using ex-vivo data on both a cylinder calibration object as well as various plastic organs.

[1]  Kurt Konolige,et al.  Projected texture stereo , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  L. Way,et al.  Causes and Prevention of Laparoscopic Bile Duct Injuries: Analysis of 252 Cases From a Human Factors and Cognitive Psychology Perspective , 2003, Annals of surgery.

[3]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[4]  Guang-Zhong Yang,et al.  Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging , 2011, Biomedical optics express.

[5]  Naoki Suzuki,et al.  Laser-scan endoscope system for intraoperative geometry acquisition and surgical robot safety management , 2006, Medical Image Anal..

[6]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[7]  Christophe Doignon,et al.  A structured light-based laparoscope with real-time organs' surface reconstruction for minimally invasive surgery , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Lena Maier-Hein,et al.  Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery , 2013, Medical Image Anal..

[9]  Armin Schneider,et al.  Time-of-Flight 3-D Endoscopy , 2009, MICCAI.

[10]  Christoph Schmalz,et al.  An endoscopic 3D scanner based on structured light , 2012, Medical Image Anal..

[11]  Michael Figl,et al.  3D Reconstruction of Internal Organ Surfaces for Minimal Invasive Surgery , 2007, MICCAI.

[12]  Guang-Zhong Yang,et al.  Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery , 2004, MICCAI.

[13]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[14]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[15]  Gregory D. Hager,et al.  Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation , 2004, MICCAI.

[16]  Yoshihiko Nakamura,et al.  Laser-pointing endoscope system for intra-operative 3D geometric registration , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[17]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Henry Fuchs,et al.  Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality , 2002, IS&T/SPIE Electronic Imaging.

[19]  H. Haneishi,et al.  Profilometry of a gastrointestinal surface by an endoscope with laser beam projection. , 1994, Optics letters.