Three-Dimensional Modeling From Endoscopic Video Using Geometric Constraints Via Feature Positioning

The endoscope is a popular imaging modality used in many preevaluations and surgical treatments, and is also one of the essential tools in minimally invasive surgery. However, regular endoscopes provide only 2-D images. Even though stereoendoscopy systems can display 3-D images, the real anatomical structure of the observed lesion is unavailable and can only be judged by the surgeon's imagination. In this paper, we present a constraint-based factorization method for reconstructing 3-D structures registered to the patient, from 2-D endoscopic images. The proposed method incorporates the geometric constraints from the tracked surgical instrument into the traditional factorization method based on frame-to-frame feature motion on the endoscopically viewed scene. Experiments with real and synthetic data demonstrate good real-scale 3-D extraction, with greater accuracy than is available from traditional methods. The reconstruction process can also be accomplished in a few seconds, making it suitable for on-line surgical applications to provide surgeons with additional 3-D shape information, critical distance monitoring and warnings.

[1]  Dirk Bartz,et al.  VIVENDI - A Virtual Ventricle Endoscopy System for Virtual Medicine , 1999 .

[2]  Elizabeth Eskow,et al.  A New Modified Cholesky Factorization , 1990, SIAM J. Sci. Comput..

[3]  Elizabeth Eskow,et al.  A Revised Modified Cholesky Factorization Algorithm , 1999, SIAM J. Optim..

[4]  Mei Han,et al.  Multiple Motion Scene Reconstruction with Uncalibrated Cameras , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Stavros G. Demos,et al.  Endoscopic method for large-depth optical imaging of interior body organs , 2002 .

[6]  Yiannis Aloimonos,et al.  Perspective approximations , 1990, Image Vis. Comput..

[7]  J. Fujimoto,et al.  In vivo endoscopic optical biopsy with optical coherence tomography. , 1997, Science.

[8]  Chao-Hung Lin,et al.  Interactive 3-D virtual colonoscopy system , 1999, IEEE Transactions on Information Technology in Biomedicine.

[9]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[10]  Bostjan Likar,et al.  3-D/2-D registration by integrating 2-D information in 3-D , 2006, IEEE Transactions on Medical Imaging.

[11]  Naoki Suzuki,et al.  Intraoperative Fast 3D Shape Recovery of Abdominal Organs in Laparoscopy , 2002, MICCAI.

[12]  Henrik Aanæs,et al.  Robust Factorization , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Dirk Bartz,et al.  Virtual voyage: interactive navigation in the human colon , 1997, SIGGRAPH.

[14]  Ivan Bricault,et al.  Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy , 1998, IEEE Transactions on Medical Imaging.

[15]  Guang-Zhong Yang,et al.  Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery , 2004, MICCAI.

[16]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Philip E. Gill,et al.  Practical optimization , 1981 .

[18]  Ève Coste-Manière,et al.  3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[19]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[20]  Nam H. Kim,et al.  Three-dimensional localization of cochlear implant electrodes using epipolar stereophotogrammetry , 2004, IEEE Transactions on Biomedical Engineering.

[21]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Christian Daul,et al.  3-D reconstruction of microcalcification clusters using stereo imaging: algorithm and mammographic unit calibration , 2005, IEEE Transactions on Biomedical Engineering.

[25]  Yung-Nien Sun,et al.  Automatic extraction and visualization of human inner structures from endoscopic image sequences , 2004, SPIE Medical Imaging.

[26]  Darius Burschka,et al.  Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery , 2005, Medical Image Anal..

[27]  S. Umeyama,et al.  Least-Squares Estimation of Transformation Parameters Between Two Point Patterns , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Florencia Almansi,et al.  Mobilizing projects in community organizations with a long-term perspective: neighbourhood credit funds in Buenos Aires, Argentina , 2008 .

[29]  Valerie Duay,et al.  A method to track cortical surface deformations using a laser range scanner , 2005, IEEE Transactions on Medical Imaging.

[30]  G. Wayne Brodland,et al.  Automated 3-D reconstruction of the surface of live early-stage amphibian embryos , 2005, IEEE Transactions on Biomedical Engineering.

[31]  K. Deguchi,et al.  Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[32]  Peter F. Sturm,et al.  Using geometric constraints through parallelepipeds for calibration and 3D modeling , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Eduard Gröller,et al.  Mastering interactive virtual Bronchioscopy on a Low—end PC , 2000 .