Virtual reality simulation in neurosurgery: technologies and evolution.

Neurosurgeons are faced with the challenge of learning, planning, and performing increasingly complex surgical procedures in which there is little room for error. With improvements in computational power and advances in visual and haptic display technologies, virtual surgical environments can now offer potential benefits for surgical training, planning, and rehearsal in a safe, simulated setting. This article introduces the various classes of surgical simulators and their respective purposes through a brief survey of representative simulation systems in the context of neurosurgery. Many technical challenges currently limit the application of virtual surgical environments. Although we cannot yet expect a digital patient to be indistinguishable from reality, new developments in computational methods and related technology bring us closer every day. We recognize that the design and implementation of an immersive virtual reality surgical simulator require expert knowledge from many disciplines. This article highlights a selection of recent developments in research areas related to virtual reality simulation, including anatomic modeling, computer graphics and visualization, haptics, and physics simulation, and discusses their implication for the simulation of neurosurgery.

[1]  Renato Pajarola,et al.  Interactive SPH simulation and rendering on the GPU , 2010, SCA '10.

[2]  John Kenneth Salisbury,et al.  Haptic Rendering: Introductory Concepts , 2004, IEEE Computer Graphics and Applications.

[3]  Ming C. Lin,et al.  A modular haptic rendering algorithm for stable and transparent 6-DOF manipulation , 2006, IEEE Transactions on Robotics.

[4]  L. O. Svaasand,et al.  OPTICAL PROPERTIES OF HUMAN BRAIN , 1983, Photochemistry and photobiology.

[5]  Anil T Ahuja,et al.  Craniotomy and Clipping of Intracranial Aneurysm in a Stereoscopic Virtual Reality Environment: Technical Note and Illustrations , 2008 .

[6]  Markus Hadwiger,et al.  GPU-Based High-Quality Volume Rendering For Virtual Environments , 2006 .

[7]  Jos Stam,et al.  Stable fluids , 1999, SIGGRAPH.

[8]  Robert Riener,et al.  Medical Model Generation , 2012 .

[9]  Ghassan Hamarneh,et al.  Efficient interactive 3D Livewire segmentation of complex objects with arbitrary topology , 2008, Comput. Medical Imaging Graph..

[10]  F. Charbel,et al.  VIRTUAL REALITY IN NEUROSURGICAL EDUCATION: PART‐TASK VENTRICULOSTOMY SIMULATION WITH DYNAMIC VISUAL AND HAPTIC FEEDBACK , 2007, Neurosurgery.

[11]  D. Louis Collins,et al.  Automatic 3‐D model‐based neuroanatomical segmentation , 1995 .

[12]  Peter Stoeter,et al.  Virtual reality system for planning minimally invasive neurosurgery. Technical note. , 2008, Journal of neurosurgery.

[13]  Jayaram K. Udupa,et al.  A 3D generalization of user-steered live-wire segmentation , 2000, Medical Image Anal..

[14]  Katherine J. Kuchenbecker,et al.  Haptic display of realistic tool contact via dynamically compensated control of a dedicated actuator , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  T. Grantcharov,et al.  Randomized clinical trial of virtual reality simulation for laparoscopic skills training , 2004, The British journal of surgery.

[16]  J. Y. C. Chen,et al.  Review of Low Frame Rate Effects on Human Performance , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[17]  Hervé Delingette,et al.  Computational Models for Image-Guided Robot-Assisted and Simulated Medical Interventions , 2006, Proceedings of the IEEE.

[18]  Blake Hannaford,et al.  Stable haptic interaction with virtual environments , 1999, IEEE Trans. Robotics Autom..

[19]  K. Miller,et al.  Constitutive model of brain tissue suitable for finite element analysis of surgical procedures. , 1999, Journal of biomechanics.

[20]  Nigel W. John,et al.  Realistic Visualization of Living Brain Tissue , 2011, MMVR.

[21]  K Miller,et al.  Mechanical properties of brain tissue in-vivo: experiment and computer simulation. , 2000, Journal of biomechanics.

[22]  David F. McAllister,et al.  Stereo computer graphics: and other true 3D technologies , 1993 .

[23]  Jaime Landman,et al.  Validating the use of the Mimic dV-trainer for robotic surgery skill acquisition among urology residents. , 2011, Urology.

[24]  Heinz-Otto Peitgen,et al.  Interactive 3D medical image segmentation with energy-minimizing implicit functions , 2011, Comput. Graph..

[25]  Dan Morris,et al.  Automatic preparation, calibration, and simulation of deformable objects , 2008, Computer methods in biomechanics and biomedical engineering.

[26]  R. Satava.  Virtual reality surgical simulator , 1993, Surgical Endoscopy.

[27]  Markus Hadwiger,et al.  High-Quality Multimodal Volume Rendering for Preoperative Planning of Neurosurgical Interventions , 2007, IEEE Transactions on Visualization and Computer Graphics.

[28]  Marvin P. Fried,et al.  From virtual reality to the operating room: The endoscopic sinus surgery simulator experiment , 2010, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[29]  John Kenneth Salisbury,et al.  Representing Fluid with Smoothed Particle Hydrodynamics in a Cranial Base Simulator , 2008, MMVR.

[30]  William E. Lorensen,et al.  Computer-assisted Interactive Three-dimensional Planning Neurosurgical Procedures , 1996 .

[31]  J. Monaghan.  Smoothed particle hydrodynamics , 2005 .

[32]  J. Z. Zhu,et al.  The finite element method , 1977 .

[33]  Hans-Peter Meinzer,et al.  Statistical shape models for 3D medical image segmentation: A review , 2009, Medical Image Anal..

[34]  N. Hata,et al.  An integrated visualization system for surgical planning and guidance using image fusion and an open MR , 2001, Journal of magnetic resonance imaging : JMRI.

[35]  Bruce J. Tromberg,et al.  In-vivo local determination of tissue optical properties , 1999, European Conference on Biomedical Optics.

[36]  Peter Y K Hwang,et al.  VIRTUAL TEMPORAL BONE: AN INTERACTIVE 3‐DIMENSIONAL LEARNING AID FOR CRANIAL BASE SURGERY , 2009, Neurosurgery.

[37]  Lukas Mroz,et al.  Advanced virtual endoscopic pituitary surgery , 2005, IEEE Transactions on Visualization and Computer Graphics.

[38]  Thomas Czech,et al.  ADVANCED VIRTUAL ENDOSCOPY FOR ENDOSCOPIC TRANSSPHENOIDAL PITUITARY SURGERY , 2006, Neurosurgery.

[39]  N. Hata,et al.  Image-guided neurosurgery at Brigham and Women's Hospital , 2006, IEEE Engineering in Medicine and Biology Magazine.

[40]  P. Bradley.  The history of simulation in medical education and possible future directions , 2006, Medical education.

[41]  Wiro J. Niessen.  Model-Based Image Segmentation for Image-Guided Interventions , 2008 .

[42]  M. Downes,et al.  Simulation in Neurosurgery: A Review of Computer-Based Simulation Environments and Their Surgical Applications , 2010, Neurosurgery.

[43]  Sébastien Ourselin,et al.  High-Speed Nonlinear Finite Element Analysis for Surgical Simulation Using Graphics Processing Units , 2008, IEEE Transactions on Medical Imaging.

[44]  Robert DiRaddo,et al.  NeuroTouch: A Physics-Based Virtual Simulator for Cranial Microneurosurgery Training , 2012, Neurosurgery.

[45]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[46]  Dinesh K. Pai,et al.  DyRT: dynamic response textures for real time deformation simulation with graphics hardware , 2002, SIGGRAPH.

[47]  Bernhard Preim,et al.  Visualization in Medicine: Theory, Algorithms, and Applications , 2007 .

[48]  R. Aggarwal,et al.  Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery , 2008, The British journal of surgery.

[49]  Christian Duriez,et al.  Realistic haptic rendering of interacting deformable objects in virtual environments , 2008, IEEE Transactions on Visualization and Computer Graphics.

[50]  Jernej Barbic,et al.  Six-DoF Haptic Rendering of Contact Between Geometrically Complex Reduced Deformable Models , 2008, IEEE Transactions on Haptics.

[51]  G. Gillies,et al.  Measurement of the force required to move a neurosurgical probe through in vivo human brain tissue , 1999, IEEE Transactions on Biomedical Engineering.

[52]  Garnette R Sutherland,et al.  Advancing neurosurgery with image-guided robotics. , 2009, Journal of neurosurgery.

[53]  John Kenneth Salisbury,et al.  The effect of sensor/actuator asymmetries in haptic interfaces , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[54]  Matthias Müller,et al.  Interactive blood simulation for virtual surgery based on smoothed particle hydrodynamics. , 2004, Technology and health care : official journal of the European Society for Engineering and Medicine.

[55]  Matthias Harders.  Surgical scene generation for virtual reality-based training in medicine , 2008 .

[56]  Richard M Satava,et al.  Virtual reality surgical simulator: the first steps. 1993. , 2006, Clinical orthopaedics and related research.

[57]  Fady Charbel,et al.  Second generation haptic ventriculostomy simulator using the ImmersiveTouch system. , 2006, Studies in health technology and informatics.

[58]  K. K. Tan,et al.  The brain: integrated three-dimensional display of MR and PET images. , 1989, Radiology.

[59]  K. Bushby,et al.  The Golden Freeway: a preliminary evaluation of a pilot study advancing information technology as a social intervention for boys with Duchenne muscular dystrophy and their families. , 2004, Health & social care in the community.

[60]  Vincenzo Liberatore,et al.  Virtual environment-based training simulator for endoscopic third ventriculostomy. , 2006, Studies in health technology and informatics.

[61]  Aljoscha Smolic,et al.  An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution , 2009, 2009 Picture Coding Symposium.

[62]  Timo Ropinski,et al.  Interactive volumetric lighting simulating scattering and shadowing , 2010, 2010 IEEE Pacific Visualization Symposium (PacificVis).

[63]  Elspeth M McDougall,et al.  Validation study of a virtual reality robotic simulator--role as an assessment tool? , 2012, The Journal of urology.

[64]  B. Dunkin,et al.  Surgical simulation: a current review , 2007, Surgical Endoscopy.

[65]  Karol Miller,et al.  Real-Time Nonlinear Finite Element Computations on GPU - Application to Neurosurgical Simulation. , 2010, Computer methods in applied mechanics and engineering.

[66]  Mariano Alcañiz Raya,et al.  Real-time deformable models for surgery simulation: a survey , 2005, Comput. Methods Programs Biomed..

[67]  R. Kikinis,et al.  Computer-assisted interactive three-dimensional planning for neurosurgical procedures. , 1996, Neurosurgery.

[68]  Clifford R. Jack,et al.  Magnetic resonance elastography of the brain , 2008, NeuroImage.

[69]  Michael Ortega-Binderberger,et al.  A Six Degree-of-Freedom God-Object Method for Haptic Display of Rigid Bodies with Surface Properties , 2007, IEEE Transactions on Visualization and Computer Graphics.

[70]  M. C. Cavusoglu,et al.  In Touch with Robotics: Neurosurgery for the Future , 2005, Neurosurgery.

[71]  Herve Delingette,et al.  Real-Time Elastic Deformations of Soft Tissues for Surgery Simulation , 1999, IEEE Trans. Vis. Comput. Graph..

[72]  Mark C Preul,et al.  Interactive stereoscopic virtual reality: a new tool for neurosurgical education. Technical note. , 2002, Journal of neurosurgery.

[73]  V Blazek,et al.  Optical properties of human brain tissue, meninges, and brain tumors in the spectral range of 200 to 900 nm. , 1987, Neurosurgery.

[74]  M. Apuzzo,et al.  Virtual Reality Surgery: Neurosurgery and the Contemporary Landscape , 2003, Neurosurgery.