RECOVERING TISSUE DEFORMATION

Recent advances in surgical robotics have provided a platform for extending the current capabilities of minimally invasive surgery by incorporating both pre-operative and intra-operative imaging data. In this tutorial paper, we introduce techniques for in vivo 3D tissue deformation recovery and tracking, based on laparoscopic or endoscopic images. These optically based techniques provide a unique opportunity for recovering surface deformation of the softtissue without the need of additional instrumentation. They can therefore be easily incorporated into the existing surgical workflow. Technically, the problem formulation is challenging due to non-rigid deformation of the tissue and instrument interaction. Current approaches and future research directions in terms of intra-operative planning and adaptive surgical navigation are explained in detail.

[1]  Michael B. Kimmey,et al.  Tethered Capsule Endoscopy, A Low-Cost and High-Performance Alternative Technology for the Screening of Esophageal Cancer and Barrett's Esophagus , 2008, IEEE Transactions on Biomedical Engineering.

[2]  Michael Figl,et al.  Non-rigid Reconstruction of the Beating Heart Surface for Minimally Invasive Cardiac Surgery , 2009, MICCAI.

[3]  Tobias Ortmaier,et al.  Motion estimation in beating heart surgery , 2005, IEEE Transactions on Biomedical Engineering.

[4]  Guang-Zhong Yang,et al.  Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[5]  Lorenzo Torresani,et al.  Tracking and modeling non-rigid objects with rank constraints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Luc Soler,et al.  Beating heart tracking in robotic surgery using 500 Hz visual servoing, model predictive control and an adaptive observer , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[7]  V. Shayani,et al.  Determination of the learning curve of the AESOP robot , 1997, Surgical Endoscopy.

[8]  Philippe Poignet,et al.  Motion prediction for tracking the beating heart , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Sebastian Thrun,et al.  Online simultaneous localization and mapping with detection and tracking of moving objects: theory and results from a ground vehicle in crowded urban areas , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[10]  Baoxin Li,et al.  Circular generalized cylinder fitting for 3D reconstruction in endoscopic imaging based on MRF , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Cameron N. Riviere,et al.  Predicting respiratory motion for active canceling during percutaneous needle insertion , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Branislav Jaramaz,et al.  A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes , 2009, International Journal of Computer Vision.

[13]  Ralph J Damiano.  Robotics in cardiac surgery: the Emperor's new clothes. , 2007, The Journal of thoracic and cardiovascular surgery.

[14]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[15]  Luc Soler,et al.  Active filtering of physiological motion in robotized surgery using predictive control , 2005, IEEE Transactions on Robotics.

[16]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[17]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..

[18]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[19]  Guang-Zhong Yang,et al.  Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures , 2005, MICCAI.

[20]  Nassir Navab,et al.  A Global Approach for Automatic Fibroscopic Video Mosaicing in Minimally Invasive Diagnosis , 2008, MICCAI.

[21]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[22]  Guang-Zhong Yang,et al.  A stereoscopic fibroscope for camera motion and 3D depth recovery during Minimally Invasive Surgery , 2009, 2009 IEEE International Conference on Robotics and Automation.

[23]  Luc Van Gool,et al.  Real-time range acquisition by adaptive structured light , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yehezkel Yeshurun,et al.  Perspective shape-from-shading by fast marching , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[25]  Gregory D. Hager,et al.  Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation , 2004, MICCAI.

[26]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[27]  Allison M. Okamura,et al.  Modeling of Tool-Tissue Interactions for Computer-Based Surgical Simulation: A Literature Review , 2008, PRESENCE: Teleoperators and Virtual Environments.

[28]  Jacques Gangloff,et al.  Physiological motion rejection in flexible endoscopy using visual servoing , 2008, ICRA.

[29]  Jacques Gangloff,et al.  Toward Robotized Beating Heart TECABG: Assessment of the Heart Dynamics Using High-Speed Vision , 2005, MICCAI.

[30]  Guang-Zhong Yang,et al.  Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery , 2006, MICCAI.

[31]  Ève Coste-Manière,et al.  3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[32]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Chao Liu,et al.  Efficient 3D Tracking for Motion Compensation in Beating Heart Surgery , 2008, MICCAI.

[34]  Guang-Zhong Yang,et al.  Illumination position estimation for 3D soft-tissue reconstruction in robotic minimally invasive surgery , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Cameron N. Riviere,et al.  Robotic Compensation of Biological Motion to Enhance Surgical Accuracy , 2006, Proceedings of the IEEE.

[36]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[37]  Steven M. Seitz,et al.  Rectified Surface Mosaics , 2007, ICCV.

[38]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[39]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[40]  Chao Liu,et al.  Deformable motion tracking of the heart surface , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[42]  Guang-Zhong Yang,et al.  i-BRUSH: A Gaze-Contingent Virtual Paintbrush for Dense 3D Reconstruction in Robotic Assisted Surgery , 2009, MICCAI.

[43]  Tatsuo Igarashi,et al.  Computer‐based endoscopic image‐processing technology for endourology and laparoscopic surgery , 2009, International journal of urology : official journal of the Japanese Urological Association.

[44]  M. de Mathelin,et al.  Comparison of visual tracking algorithms on in vivo sequences for robot-assisted flexible endoscopic surgery , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[45]  Chia-Hsiang Wu,et al.  Three-Dimensional Modeling From Endoscopic Video Using Geometric Constraints Via Feature Positioning , 2007, IEEE Transactions on Biomedical Engineering.

[46]  Michael Figl,et al.  3D Reconstruction of Internal Organ Surfaces for Minimal Invasive Surgery , 2007, MICCAI.

[47]  Guang-Zhong Yang,et al.  A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery , 2007, MICCAI.

[48]  K. Deguchi,et al.  Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[49]  Guang-Zhong Yang,et al.  Soft tissue tracking for minimally invasive surgery: learning local deformation online. , 2008, Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention.

[50]  Gregory D. Hager,et al.  Real-Time Endoscopic Mosaicking , 2006, MICCAI.

[51]  Guang-Zhong Yang,et al.  Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery , 2004, MICCAI.

[52]  Armin Schneider,et al.  Time-of-Flight 3-D Endoscopy , 2009, MICCAI.

[53]  T. Peters.  Image-guidance for surgical procedures , 2006, Physics in medicine and biology.

[54]  Guang-Zhong Yang,et al.  Gaze contingent articulated robot control for robot assisted minimally invasive surgery , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[55]  Tobias Ortmaier,et al.  Tracking local motion on the beating heart , 2002, SPIE Medical Imaging.

[56]  Castaneda V,et al.  Reconstructing the Esophagus Surface from Endoscopic Image Sequences , 2009 .

[57]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[58]  Philippe C. Cattin,et al.  Endoscopic Navigation for Minimally Invasive Suturing , 2007, MICCAI.

[59]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[60]  Philippe Poignet,et al.  Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis , 2007, Biomed. Signal Process. Control..

[61]  Gerd Hirzinger,et al.  OPTICAL FLOW TO ANALYSE STABILISED IMAGES OF THE BEATING HEART , 2006 .

[62]  Darius Burschka,et al.  Scale-Invariant Registration of Monocular Endoscopic Images to CT-Scans for Sinus Surgery , 2004, MICCAI.

[63]  J. Triboulet,et al.  Texture Features Selection for Visual Servoing of the Beating Heart , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[64]  D. Wolf,et al.  Combination of panoramic and fluorescence endoscopic images to obtain tumor spatial distribution information useful for bladder cancer detection , 2007, European Conference on Biomedical Optics.

[65]  Naoki Suzuki,et al.  Laser-scan endoscope system for intraoperative geometry acquisition and surgical robot safety management , 2006, Medical Image Anal..

[66]  Carlos Henrique Quartucci Forster,et al.  Towards 3D reconstruction of endoscope images using shape from shading , 2000, Proceedings 13th Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00878).

[67]  Pierre Graebling,et al.  Robust Structured Light Coding for 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[68]  Guang-Zhong Yang,et al.  Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  Alexander Behrens.  Creating Panoramic Images for Bladder Fluorescence Endoscopy , 2008 .

[70]  Christian Daul,et al.  Mosaicing of Bladder Endoscopic Image Sequences: Distortion Calibration and Registration Algorithm , 2008, IEEE Transactions on Biomedical Engineering.

[71]  Guang-Zhong Yang,et al.  Gaze-Contingent Soft Tissue Deformation Tracking for Minimally Invasive Robotic Surgery , 2005, MICCAI.

[72]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[73]  H. Haneishi,et al.  Profilometry of a gastrointestinal surface by an endoscope with laser beam projection. , 1994, Optics letters.

[74]  Brian L. Davies,et al.  Active-Constraint Robotics for Surgery , 2006, Proceedings of the IEEE.

[75]  M. Mack,et al.  Minimally invasive and robotic surgery. , 2001, JAMA.

[76]  Peter Burger,et al.  Differential algorithm for the determination of shape from shading using a point light source , 1992, Image Vis. Comput..

[77]  Yuan-Fang Wang,et al.  Toward automated model building from video in computer-assisted diagnoses in colonoscopy , 2007, SPIE Medical Imaging.

[78]  Olivier D. Faugeras,et al.  Shape from shading: a well-posed problem? , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[79]  A. Park,et al.  A technique for real-time digital measurements in laparoscopic surgery , 2004, Surgical Endoscopy And Other Interventional Techniques.

[80]  Guang-Zhong Yang,et al.  Belief Propagation for Depth Cue Fusion in Minimally Invasive Surgery , 2008, MICCAI.