In vivo and in situ image guidance and modelling in robotic assisted surgery

Abstract The increase in the use of robotic assisted surgery has provided benefits to both the patient and the surgeon. There are possibilities for training and surgical planning and guidance within this framework but work is needed for this to reach a patient-specific level. Medical images obtained prior to surgery are used clinically to plan operations and there has been much research in their use to build models, giving further knowledge of tissue and organ structures. The incorporation of this data into the surgical environment is a recent development, assisting the surgeon with orientation information, but motion due to respiration or changes to morphology due to intervention begets a need for the incorporation of real-time imaging into the modelling. The current work on pre-operative and intra-operative imaging and modelling methods for robotic assisted surgery has to be reviewed and then the Image Constrained Biomechanical Modelling has to be introduced, whereby patient-specific biomechanical modelling is based on the underlying medical images, reducing the computational complexity involved with traditional biomechani-cal modelling. This article concludes with a discussion of the possible and promising future directions of this work.

[1]  Guang-Zhong Yang,et al.  Assessment of Perceptual Quality for Gaze-Contingent Motion Stabilization in Robotic Assisted Minimally Invasive Surgery , 2007, MICCAI.

[2]  Christos E. Constantinou,et al.  Real time robotic tactile sensor system for the determination of the physical properties of biomaterials , 2004 .

[3]  R. Kikinis,et al.  Integration of interventional MRI with computer‐assisted surgery , 2001, Journal of magnetic resonance imaging : JMRI.

[4]  T. Marwick.  Measurement of strain and strain rate by echocardiography: ready for prime time? , 2006, Journal of the American College of Cardiology.

[5]  Guang-Zhong Yang,et al.  Stabilization of Image Motion for Robotic Assisted Beating Heart Surgery , 2007, MICCAI.

[6]  Philippe Poignet,et al.  Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis , 2007, Biomed. Signal Process. Control..

[7]  Nicholas Ayache,et al.  A New Framework for Fusing Stereo Images with Volumetric Medical Images , 1995, CVRMed.

[8]  Guang-Zhong Yang,et al.  Belief Propagation for Depth Cue Fusion in Minimally Invasive Surgery , 2008, MICCAI.

[9]  Jing Ren,et al.  Dynamic 3-D Virtual Fixtures for Minimally Invasive Beating Heart Procedures , 2008, IEEE Transactions on Medical Imaging.

[10]  Hervé Delingette,et al.  Realistic simulation of the 3-D growth of brain tumors in MR images coupling diffusion with biomechanical deformation , 2005, IEEE Transactions on Medical Imaging.

[11]  Martin J. Yaffe,et al.  Biomechanical 3-D finite element modeling of the human breast using MRI data , 2001, IEEE Transactions on Medical Imaging.

[12]  Sebastian Wachsmann-Hogiu,et al.  Chemical analysis in vivo and in vitro by Raman spectroscopy--from single cells to humans. , 2009, Current opinion in biotechnology.

[13]  O. Skrinjar,et al.  A stereo-guided biomechanical model for volumetric deformation analysis , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[14]  B. Gersh,et al.  Cardiac cell repair therapy: a clinical perspective. , 2009, Mayo Clinic proceedings.

[15]  Keith D. Paulsen,et al.  Data-Guided Brain Deformation Modeling: Evaluation of a 3-D Adjoint Inversion Method in Porcine Studies , 2006, IEEE Transactions on Biomedical Engineering.

[16]  Lawrence H. Staib,et al.  Physical model-based non-rigid registration incorporating statistical shape information , 2000, Medical Image Anal..

[17]  David J. Hawkes,et al.  Validation of nonrigid image registration using finite-element methods: application to breast MR images , 2003, IEEE Transactions on Medical Imaging.

[18]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..

[19]  R. Weissleder,et al.  Imaging in the era of molecular oncology , 2008, Nature.

[20]  J. Fujimoto,et al.  Optical coherence tomography of the human retina. , 1995, Archives of ophthalmology.

[21]  David J. Hawkes,et al.  A Framework for Image-Guided Breast Surgery , 2006, MIAR.

[22]  Guang-Zhong Yang,et al.  pq-space Based Non-Photorealistic Rendering for Augmented Reality , 2007, MICCAI.

[23]  Yudong Zhu,et al.  A spatiotemporal model of cyclic kinematics and its application to analyzing nonrigid motion with MR velocity images , 1999, IEEE Transactions on Medical Imaging.

[24]  S Hassfeld,et al.  Semiautomated registration using new markers for assessing the accuracy of a navigation system. , 2002, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[25]  H. Wen,et al.  DENSE: displacement encoding with stimulated echoes in cardiac functional MRI. , 1999, Journal of magnetic resonance.

[26]  James S. Duncan,et al.  A model-based integrated approach to track myocardial deformation using displacement and velocity constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Su-Lin Lee,et al.  Contractile Analysis with Kriging Based on MR Myocardial Velocity Imaging , 2008, MICCAI.

[28]  L. Axel,et al.  MR imaging of motion with spatial modulation of magnetization. , 1989, Radiology.

[29]  Guang-Zhong Yang,et al.  Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures , 2005, MICCAI.

[30]  D. Firmin,et al.  Measurement of Flow with NMR Imaging Using a Gradient Pulse and Phase Difference Technique , 1984, Journal of computer assisted tomography.

[31]  T. Shiina,et al.  Fast Reconstruction of Tissue Elastic Modulus Image by Ultrasound , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[32]  Christos Davatzikos,et al.  Nonlinear elastic registration of brain images with tumor pathology using a biomechanical model [MRI] , 1999, IEEE Transactions on Medical Imaging.

[33]  Stephane Cotin,et al.  EP4A: Software and Computer Based Simulator Research: Development and Outlook SOFA—An Open Source Framework for Medical Simulation , 2007, MMVR.

[34]  Hervé Delingette,et al.  Computational Models for Image-Guided Robot-Assisted and Simulated Medical Interventions , 2006, Proceedings of the IEEE.

[35]  P. Hunter,et al.  Integration from proteins to organs: the Physiome Project , 2003, Nature Reviews Molecular Cell Biology.

[36]  Robert L. Galloway,et al.  Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements , 2005, IEEE Transactions on Medical Imaging.

[37]  Kawal S. Rhode,et al.  An Adaptive and Predictive Respiratory Motion Model for Image-Guided Interventions: Theory and First Clinical Application , 2009, IEEE Transactions on Medical Imaging.

[38]  The in vivo characterization of the mechanical properties of human skin by MRI and indentation techniques , 2007 .

[39]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Wei Jin,et al.  On Some Recent Advances in Multimodal Surgery Simulation: A Hybrid Approach to Surgical Cutting and the Use of Video Images for Enhanced Realism , 2007, PRESENCE: Teleoperators and Virtual Environments.

[41]  R. Kiesslich,et al.  Confocal endomicroscopy: in vivo diagnosis of neoplastic lesions of the gastrointestinal tract. , 2008, Anticancer research.

[42]  Ralph J Damiano.  Robotics in cardiac surgery: the Emperor's new clothes. , 2007, The Journal of thoracic and cardiovascular surgery.

[43]  Guang-Zhong Yang,et al.  Virtual tagging: numerical considerations and phantom validation , 2002, IEEE Transactions on Medical Imaging.

[44]  C Veyrat,et al.  Tissue Doppler, strain, and strain rate echocardiography for the assessment of left and right systolic ventricular function , 2003, Heart.

[45]  David J. Hawkes,et al.  Design and evaluation of a system for microscope-assisted guided interventions (MAGI) , 2000 .

[46]  Peter Kazanzides,et al.  Anatomy-based registration of CT-scan and intraoperative X-ray images for guiding a surgical robot , 1998, IEEE Transactions on Medical Imaging.

[47]  S R Arridge,et al.  Recent advances in diffuse optical imaging , 2005, Physics in medicine and biology.

[48]  J. Fujimoto,et al.  Optical Coherence Tomography , 1991 .

[49]  W. Eric L. Grimson,et al.  An Anisotropic Material Model for Image Guided Neurosurgery , 2004, MICCAI.

[50]  Tomas Lozano-Perez,et al.  An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization , 1996 .

[51]  G. Gelikonov,et al.  Endoscopic OCT with forward‐looking probe: clinical studies in urology and gastroenterology , 2008, Journal of biophotonics.

[52]  Guang-Zhong Yang,et al.  Predictive cardiac motion modeling and correction with partial least squares regression , 2004, IEEE Transactions on Medical Imaging.

[53]  D. Weiss,et al.  Cell therapy approaches for lung diseases: current status. , 2009, Current opinion in pharmacology.

[54]  Christopher Nimsky,et al.  Estimating Mechanical Brain Tissue Properties with Simulation and Registration , 2004, MICCAI.

[55]  Nassir Navab,et al.  The Virtual Mirror: A New Interaction Paradigm for Augmented Reality Environments , 2009, IEEE Transactions on Medical Imaging.

[56]  Guang-Zhong Yang,et al.  A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery , 2007, MICCAI.

[57]  Karol Miller,et al.  Patient-specific model of brain deformation: application to medical image registration. , 2007, Journal of biomechanics.

[58]  Silvia Sanduleanu,et al.  In vivo diagnosis and classification of colorectal neoplasia by chromoendoscopy-guided confocal laser endomicroscopy. , 2010, Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association.

[59]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[60]  Gregory J. Klein,et al.  Forward Deformation of PET Volumes Using Non-uniform Elastic Material Constraints , 1999, IPMI.