Interactive simulation of surgical needle insertion and steering

We present algorithms for simulating and visualizing the insertion and steering of needles through deformable tissues for surgical training and planning. Needle insertion is an essential component of many clinical procedures such as biopsies, injections, neurosurgery, and brachytherapy cancer treatment. The success of these procedures depends on accurate guidance of the needle tip to a clinical target while avoiding vital tissues. Needle insertion deforms body tissues, making accurate placement difficult. Our interactive needle insertion simulator models the coupling between a steerable needle and deformable tissue. We introduce (1) a novel algorithm for local remeshing that quickly enforces the conformity of a tetrahedral mesh to a curvilinear needle path, enabling accurate computation of contact forces, (2) an efficient method for coupling a 3D finite element simulation with a 1D inextensible rod with stick-slip friction, and (3) optimizations that reduce the computation time for physically based simulations. We can realistically and interactively simulate needle insertion into a prostate mesh of 13,375 tetrahedra and 2,763 vertices at a 25 Hz frame rate on an 8-core 3.0 GHz Intel Xeon PC. The simulation models prostate brachytherapy with needles of varying stiffness, steering needles around obstacles, and supports motion planning for robotic needle insertion. We evaluate the accuracy of the simulation by comparing against real-world experiments in which flexible, steerable needles were inserted into gel tissue phantoms.

[1]  E. Cuthill,et al.  Reducing the bandwidth of sparse symmetric matrices , 1969, ACM '69.

[2]  Hiroshi Akima,et al.  A New Method of Interpolation and Smooth Curve Fitting Based on Local Procedures , 1970, JACM.

[3]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[4]  J. C. Simo,et al.  A finite strain beam formulation. The three-dimensional dynamic problem. Part I , 1985 .

[5]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[6]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[7]  Demetri Terzopoulos,et al.  Modeling inelastic deformation: viscolelasticity, plasticity, fracture , 1988, SIGGRAPH.

[8]  S. Antman.  Nonlinear problems of elasticity , 1994 .

[9]  V. Parthasarathy,et al.  A comparison of tetrahedron quality measures , 1994 .

[10]  Robert Davis Cook,et al.  Finite Element Modeling for Stress Analysis , 1995 .

[11]  D. Baraff.  An Introduction to Physically Based Modeling: Rigid Body Simulation I—Unconstrained Rigid Body Dynamics , 1997 .

[12]  T. Krouskop,et al.  Elastic Moduli of Breast and Prostate Tissues under Compression , 1998, Ultrasonic imaging.

[13]  Jonathan Richard Shewchuk,et al.  Tetrahedral mesh generation by Delaunay refinement , 1998, SCG '98.

[14]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[15]  Jessica K. Hodgins,et al.  Graphical modeling and animation of brittle fracture , 1999, SIGGRAPH.

[16]  Mathieu Desbrun,et al.  Adaptive simulation of soft bodies in real-time , 2000, Proceedings Computer Animation 2000.

[17]  R. Taschereau,et al.  Seed misplacement and stabilizing needles in transperineal permanent prostate implants. , 2000, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[18]  Han-Wen Nienhuys,et al.  A Surgery Simulation Supporting Cuts and Finite Element Deformation , 2001, MICCAI.

[19]  Elmar Schömer,et al.  A Virtual Environment for Interactive Assembly Simulation: From Rigid Bodies to Deformable Cables , 2001 .

[20]  Leonard McMillan,et al.  Stable real-time deformations , 2002, SCA '02.

[21]  R. Satava,et al.  Virtual Reality Training Improves Operating Room Performance: Results of a Randomized, Double-Blinded Study , 2002, Annals of surgery.

[22]  Dinesh K. Pai,et al.  STRANDS: Interactive Simulation of Thin Solids using Cosserat Models , 2002, Comput. Graph. Forum.

[23]  P. Maurette,et al.  [To err is human: building a safer health system]. , 2002, Annales francaises d'anesthesie et de reanimation.

[24]  Allison M. Okamura,et al.  Modeling of needle insertion forces for robot-assisted percutaneous therapy , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[25]  Jakob Andreas Bærentzen,et al.  Computing discrete signed distance fields from triangle meshes , 2002 .

[26]  Jessica K. Hodgins,et al.  Graphical modeling and animation of ductile fracture , 2002, SIGGRAPH.

[27]  Septimiu E. Salcudean,et al.  Needle insertion modelling and simulation , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[28]  R. LeVeque.  Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[29]  Han-Wen Nienhuys,et al.  Interactive needle insertions in 3D nonlinear material , 2003 .

[30]  Hervé Delingette,et al.  Non-linear anisotropic elasticity for real-time surgery simulation , 2003, Graph. Model..

[31]  Ronald Fedkiw,et al.  Finite volume methods for the simulation of skeletal muscle , 2003, SCA '03.

[32]  Christian Laugier,et al.  Simulating soft tissue cutting using finite element models , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[33]  Ken Goldberg,et al.  Simulating needle insertion and radioactive seed implantation for prostate brachytherapy. , 2003, Studies in health technology and informatics.

[34]  Allison M. Okamura,et al.  Robotic needle insertion: effects of friction and needle geometry , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[35]  D. Cao,et al.  Three-dimensional nonlinear dynamics of slender structures: Cosserat rod element approach , 2004, math/0410286.

[36]  James F. O'Brien,et al.  A method for animating viscoelastic fluids , 2004, ACM Trans. Graph..

[37]  Xiaogang Wang,et al.  A virtual reality based 3D real-time interactive brachytherapy simulation of needle insertion and seed implantation , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[38]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[39]  Han-Wen Nienhuys,et al.  A computational technique for interactive needle insertions in 3D nonlinear material , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[40]  Yasushi Ito,et al.  Reliable Isotropic Tetrahedral Mesh Generation Based on an Advancing Front Method , 2004, IMR.

[41]  Ronald Fedkiw,et al.  Invertible finite elements for robust simulation of large deformation , 2004, SCA '04.

[42]  S. Shankar Sastry,et al.  GiPSi: An Open Source/Open Architecture Software Development Framework for Surgical Simulation , 2004, ISMS.

[43]  Markus H. Gross,et al.  Interactive Virtual Materials , 2004, Graphics Interface.

[44]  R. Satava,et al.  Virtual Reality Simulation for the Operating Room: Proficiency-Based Training as a Paradigm Shift in Surgical Skills Training , 2005, Annals of surgery.

[45]  Richard M Satava,et al.  Identification and reduction of surgical error using simulation , 2005, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[46]  Allison M. Okamura,et al.  A Velocity-Dependent Model for Needle Insertion in Soft Tissue , 2005, MICCAI.

[47]  Septimiu E. Salcudean,et al.  Needle steering and motion planning in soft tissues , 2005, IEEE Transactions on Biomedical Engineering.

[48]  Robert J. Webster,et al.  Design Considerations for Robotic Needle Steering , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[49]  Allison M. Okamura,et al.  Planning for Steerable Bevel-tip Needle Insertion Through 2D Soft Tissue with Obstacles , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[50]  C. Paloc,et al.  Online Remeshing for Soft Tissue Simulation in Surgical Training , 2006, IEEE Computer Graphics and Applications.

[51]  Elmar Schömer,et al.  Interactive simulation of one-dimensional flexible parts , 2006, SPM '06.

[52]  Jin Seob Kim,et al.  Nonholonomic Modeling of Needle Steering , 2006, Int. J. Robotics Res..

[53]  Russell H. Taylor,et al.  A Perspective on Medical Robotics , 2006, Proceedings of the IEEE.

[54]  Marie-Paule Cani,et al.  Super-helices for predicting the dynamics of natural hair , 2006, SIGGRAPH 2006.

[55]  Maud Marchal,et al.  Simulating Prostate Surgical Procedures with a Discrete Soft Tissue Model , 2006, VRIPHYS.

[56]  Orcun Goksel,et al.  3D simulation of needle-tissue interaction with application to prostate brachytherapy , 2006, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[57]  Frank Tendick,et al.  GiPSi: a framework for open source/open architecture software development for organ-level surgical simulation , 2006, IEEE Transactions on Information Technology in Biomedicine.

[58]  Jonathan Richard Shewchuk,et al.  Aggressive Tetrahedral Mesh Improvement , 2007, IMR.

[59]  George M. Turkiyyah,et al.  A physically-based framework for real-time haptic cutting and interaction with 3D continuum models , 2007, Symposium on Solid and Physical Modeling.

[60]  Ehsan Dehghan,et al.  Needle Insertion Point and Orientation Optimization in Non-linear Tissue with Application to Brachytherapy , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[61]  Stephane Cotin,et al.  EP4A: Software and Computer Based Simulator Research: Development and Outlook SOFA—An Open Source Framework for Medical Simulation , 2007, MMVR.

[62]  Rajni V. Patel,et al.  Needle insertion into soft tissue: a survey. , 2007, Medical engineering & physics.

[63]  Jonathan Richard Shewchuk,et al.  Isosurface stuffing: fast tetrahedral meshes with good dihedral angles , 2007, ACM Trans. Graph..

[64]  Ronald Fedkiw,et al.  Volume conserving finite element simulations of deformable models , 2007, ACM Trans. Graph..

[65]  Ronald Fedkiw,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2007) Hybrid Simulation of Deformable Solids , 2022 .

[66]  J. Spillmann,et al.  CoRdE: Cosserat rod elements for the dynamic simulation of one-dimensional elastic objects , 2007, SCA '07.

[67]  Adam W. Bargteil,et al.  A finite element method for animating large viscoplastic flow , 2007, ACM Trans. Graph..

[68]  Thierry Siméon,et al.  The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty , 2007, Robotics: Science and Systems.

[69]  Kenneth Y. Goldberg,et al.  Motion Planning in Medicine: Optimization and Simulation Algorithms for Image-Guided Procedures , 2008, Springer Tracts in Advanced Robotics.

[70]  Eitan Grinspun,et al.  Discrete elastic rods , 2008, ACM Trans. Graph..

[71]  Roummel F. Marcia,et al.  On solving sparse symmetric linear systems whose definiteness is unknown , 2008 .

[72]  V. Kallem,et al.  Integrated planning and image-guided control for planar needle steering , 2008, 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[73]  Andrew M. Stuart,et al.  A First Course in Continuum Mechanics: Bibliography , 2008 .

[74]  Greg Turk,et al.  Fast viscoelastic behavior with thin features , 2008, ACM Trans. Graph..

[75]  Nigel W. John,et al.  Simulation of ultrasound guided needle puncture using patient specific data with 3D textures and volume haptics , 2008, Comput. Animat. Virtual Worlds.

[76]  Matthias Teschner,et al.  An Adaptive Contact Model for the Robust Simulation of Knots , 2008, Comput. Graph. Forum.

[77]  James F. O'Brien,et al.  Interactive simulation of surgical needle insertion and steering , 2009, ACM Trans. Graph..

[78]  C. Duriez,et al.  Suturing simulation based on complementarity constraints , 2009 .

[79]  Florence Bertails,et al.  Linear Time Super‐Helices , 2009, Comput. Graph. Forum.

[80]  Kenneth Y. Goldberg,et al.  Guiding medical needles using single-point tissue manipulation , 2009, 2009 IEEE International Conference on Robotics and Automation.

[81]  Mathieu Desbrun,et al.  Numerical coarsening of inhomogeneous elastic materials , 2009, SIGGRAPH 2009.

[82]  Allison M. Okamura,et al.  Feedback control for steering needles through 3D deformable tissue using helical paths , 2009, Robotics: Science and Systems.

[83]  James F. O'Brien,et al.  Dynamic local remeshing for elastoplastic simulation , 2010, ACM Trans. Graph..