An Augmented Reality Haptic Training Simulator for Spinal Needle Procedures

This paper presents the prototype for an augmented reality haptic simulation system with potential for spinal needle insertion training. The proposed system is composed of a torso mannequin, a MicronTracker2 optical tracking system, a PHANToM haptic device, and a graphical user interface to provide visual feedback. The system allows users to perform simulated needle insertions on a physical mannequin overlaid with an augmented reality cutaway of patient anatomy. A tissue model based on a finite-element model provides force during the insertion. The system allows for training without the need for the presence of a trained clinician or access to live patients or cadavers. A pilot user study demonstrates the potential and functionality of the system.

[1]  Allison M. Okamura,et al.  Force modeling for needle insertion into soft tissue , 2004, IEEE Transactions on Biomedical Engineering.

[2]  Septimiu E. Salcudean,et al.  Interactive simulation of needle insertion models , 2005, IEEE Transactions on Biomedical Engineering.

[3]  Stephane Cotin,et al.  A hybrid elastic model for real-time cutting, deformations, and force feedback for surgery training and simulation , 2000, The Visual Computer.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  D. Gaba.  The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[6]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[7]  Kevin Kunkler,et al.  The role of medical simulation: an overview , 2006, The international journal of medical robotics + computer assisted surgery : MRCAS.

[8]  Sim Heng Ong,et al.  Haptics in computer-mediated simulation: Training in vertebroplasty surgery , 2006 .

[9]  Nigel W. John,et al.  The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art , 2011, IEEE Transactions on Haptics.

[10]  Anne-Sophie Nyssen,et al.  A Comparison of the Training Value of Two Types of Anesthesia Simulators: Computer Screen-Based and Mannequin-Based Simulators , 2002, Anesthesia and analgesia.

[11]  Luc Soler,et al.  An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases , 2009, Medical Image Anal..

[12]  Rajni V. Patel,et al.  Needle insertion into soft tissue: a survey. , 2007, Medical engineering & physics.

[13]  Marco Pulimeno,et al.  An Augmented Reality Application for Minimally Invasive Surgery , 2008 .

[14]  Alan Watt,et al.  Advanced animation and rendering techniques - theory and practice , 1992 .

[15]  Nassir Navab,et al.  Real-Time Simulation of Medical Ultrasound from CT Images , 2008, MICCAI.

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  D. Brenner,et al.  Computed tomography--an increasing source of radiation exposure. , 2007, The New England journal of medicine.

[18]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[19]  Robert Riener,et al.  An Augmented Reality Delivery Simulator for Medical Training , 2004 .

[20]  Nigel W. John,et al.  Simulation of ultrasound guided needle puncture using patient specific data with 3D textures and volume haptics , 2008, Comput. Animat. Virtual Worlds.

[21]  K. Naemura,et al.  Needle Insertion Test by Porcine Ligamentum Flavum , 2009 .

[22]  Purang Abolmaesumi,et al.  A haptics-based ultrasound examination training system , 2004, CARS.

[23]  K. Hashtrudi-Zaad,et al.  Dynamic parameter identification and analysis of a PHANToM haptic device , 2005, Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005..

[24]  J B Cooper,et al.  A brief history of the development of mannequin simulators for clinical education and training , 2004, Quality and Safety in Health Care.

[25]  Jenny Dankelman,et al.  Visual force feedback in laparoscopic training , 2011, Surgical Endoscopy.

[26]  Stephane Cotin,et al.  Constraint-Based Haptic Rendering of Multirate Compliant Mechanisms , 2011, IEEE Transactions on Haptics.

[27]  Gene H. Golub,et al.  Matrix computations , 1983 .

[28]  Ryutaro Kawano,et al.  Mannequin Simulation Improves the Confidence of Medical Students Performing Tube Thoracostomy: A Prospective, Controlled Trial , 2010, The American surgeon.

[29]  M. Bro-Nielsen,et al.  Finite element modeling in surgery simulation , 1998, Proc. IEEE.

[30]  A. Nachemson,et al.  Some mechanical properties of the third human lumbar interlaminar ligament (ligamentum flavum). , 1968, Journal of biomechanics.

[31]  Fernando Bello,et al.  Virtual Reality Simulation of Liver Biopsy with a Respiratory Component , 2011 .

[32]  M. Sato.  [Mechanical properties of living tissues]. , 1986, Iyo denshi to seitai kogaku. Japanese journal of medical electronics and biological engineering.

[33]  Franck Vidal,et al.  Simulation of ultrasound guided needle puncture using patient specific data with 3D textures and volume haptics , 2008 .

[34]  Ekkehard Euler,et al.  Real-time Volume Rendering for High Quality Visualization in Augmented Reality , 2008 .

[35]  W. O’Brien,et al.  Young's modulus measurements of soft tissues with application to elasticity imaging , 1996, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[36]  Septimiu E. Salcudean,et al.  Needle insertion modeling and simulation , 2003, IEEE Trans. Robotics Autom..

[37]  Yongde Zhang,et al.  Evaluation of robotic needle insertion in conjunction with in vivo manual insertion in the operating room , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[38]  Berk Gonenc,et al.  Force control with hybrid actuator for virtual needle insertion , 2011, 2011 IEEE World Haptics Conference.

[39]  Penny Jennett,et al.  Learning gains derived from a high-fidelity mannequin-based simulation in the pediatric emergency department. , 2006, Journal of the Formosan Medical Association = Taiwan yi zhi.

[40]  James F. O'Brien,et al.  Interactive simulation of surgical needle insertion and steering , 2009, ACM Trans. Graph..

[41]  Nassir Navab,et al.  Advanced training methods using an Augmented Reality ultrasound simulator , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[42]  Gabor Fichtinger,et al.  Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine , 2012, Medical Image Anal..

[43]  Luis Ibáñez,et al.  The ITK Software Guide , 2005 .

[44]  J. Ophir,et al.  IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control , 2008, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control.

[45]  Fernando Bello,et al.  A prototype percutaneous transhepatic cholangiography training simulator with real-time breathing motion , 2009, International Journal of Computer Assisted Radiology and Surgery.

[46]  J. Edward Colgate,et al.  Factors affecting the Z-Width of a haptic display , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[47]  Jin Seob Kim,et al.  Nonholonomic Modeling of Needle Steering , 2006, Int. J. Robotics Res..

[48]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  B. T. Sitzman,et al.  The Effects of Needle Type, Gauge, and Tip Bend on Spinal Needle Deflection , 1996, Anesthesia and analgesia.

[50]  T. A. Thomas,et al.  Simulation of resistance forces acting on surgical needles , 1997, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[51]  Gábor Székely,et al.  Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics , 2009, IEEE Transactions on Visualization and Computer Graphics.

[52]  K. Huebner.  The finite element method for engineers , 1975 .

[53]  Annette Aboulafia,et al.  Testing Haptic Sensations for Spinal Anesthesia , 2010, Regional Anesthesia & Pain Medicine.

[54]  Orcun Goksel,et al.  Haptic Simulator for Prostate Brachytherapy with Simulated Ultrasound , 2010, ISMBS.

[55]  Purang Abolmaesumi,et al.  A Framework for the Design of a Novel Haptic-Based Medical Training Simulator , 2008, IEEE Transactions on Information Technology in Biomedicine.

[56]  Stephane Cotin,et al.  Interactive Simulation of Flexible Needle Insertions Based on Constraint Models , 2009, MICCAI.

[57]  Derek R. Magee,et al.  An augmented reality simulator for ultrasound guided needle placement training , 2007, Medical & Biological Engineering & Computing.

[58]  Nassir Navab,et al.  Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[59]  Frank Tendick,et al.  A Critical Study of the Mechanical and Electrical Properties of the PHANToM Haptic Interface and Improvements for Highperformance Control , 2002, Presence: Teleoperators & Virtual Environments.

[60]  P. Gorman,et al.  A prototype haptic lumbar puncture simulator. , 2000, Studies in health technology and informatics.

[61]  Fernando Bello,et al.  Simulation: Moving from Technology Challenge to Human Factors Success , 2012, CardioVascular and Interventional Radiology.

[62]  J. Edward Colgate,et al.  Implementation of stiff virtual walls in force-reflecting interfaces , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[63]  Ehsan Dehghan,et al.  Needle Insertion Study Using Ultrasound-Based 2D Motion Tracking , 2008, MICCAI.

[64]  Keyvan Hashtrudi-Zaad,et al.  Development of a dynamic model for bevel-tip flexible needle insertion into soft tissues , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[65]  R. Waters,et al.  An in vitro study of normal and scoliotic interspinous ligaments. , 1973, Journal of biomechanics.

[66]  Heinz Handels,et al.  Virtual Reality Based Training of Lumbar Punctures Using a 6DOF Haptic Device , 2007 .

[67]  Laehyun Kim,et al.  Needle Insertion Simulator with Haptic Feedback , 2011, HCI.

[68]  Nassir Navab,et al.  Depth Perception - A Major Issue in Medical AR: Evaluation Study by Twenty Surgeons , 2006, MICCAI.

[69]  Toshikatsu Washio,et al.  A Model for Relations between Needle Deflection, Force, and Thickness on Needle Penetration , 2001, MICCAI.

[70]  Stefan Wesarg,et al.  VR-Based Simulators for Training in Minimally Invasive Surgery , 2007, IEEE Computer Graphics and Applications.

[71]  John Magill,et al.  Multi-axis Mechanical Simulator for Epidural Needle Insertion , 2004, ISMS.

[72]  Pheng-Ann Heng,et al.  Learning Ultrasound-Guided Needle Insertion Skills through an Edutainment Game , 2010, Trans. Edutainment.

[73]  C Coolens,et al.  Calibration of CT Hounsfield units for radiotherapy treatment planning of patients with metallic hip prostheses: the use of the extended CT-scale. , 2003, Physics in medicine and biology.

[74]  J B Ra,et al.  Spine needle biopsy simulator using visual and force feedback. , 2002, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[75]  Henry Fuchs,et al.  Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms , 2002, Medical Image Anal..