Proof of Concept: Wearable Augmented Reality Video See-Through Display for Neuro-Endoscopy

In mini-invasive surgery and in endoscopic procedures, the surgeon operates without a direct visualization of the patient’s anatomy. In image-guided surgery, solutions based on wearable augmented reality (AR) represent the most promising ones. The authors describe the characteristics that an ideal Head Mounted Display (HMD) must have to guarantee safety and accuracy in AR-guided neurosurgical interventions and design the ideal virtual content for guiding crucial task in neuro endoscopic surgery. The selected sequence of AR content to obtain an effective guidance during surgery is tested in a Microsoft Hololens based app.

[1]  R. P. Gaston,et al.  A System for Microscope-Assisted Guided Interventions , 2000, Stereotactic and Functional Neurosurgery.

[2]  Fabrizio Cutolo,et al.  Video see through AR head-mounted display for medical procedures , 2014, International Symposium on Mixed and Augmented Reality.

[3]  A. Meola,et al.  Augmented reality in neurosurgery: a systematic review , 2017, Neurosurgical Review.

[4]  Ron Kikinis,et al.  Comparison of different registration methods for surgical navigation in cranio-maxillofacial surgery. , 2008, Journal of cranio-maxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery.

[5]  D. Louis Collins,et al.  The state of the art of visualization in mixed reality image guided surgery , 2013, Comput. Medical Imaging Graph..

[6]  Rosanna Maria Viglialoro,et al.  A physical patient specific simulator for cholecystectomy training , 2012 .

[7]  Weiwei Deng,et al.  Easy-to-Use Augmented Reality Neuronavigation Using a Wireless Tablet PC , 2013, Stereotactic and Functional Neurosurgery.

[8]  Fabrizio Cutolo.  Augmented Reality in Image-Guided Surgery , 2019, Encyclopedia of Computer Graphics and Games.

[9]  Osamu Ukimura,et al.  Image-fusion, augmented reality, and predictive surgical navigation. , 2009, The Urologic clinics of North America.

[10]  Marvin J. Slepian,et al.  Augmented Reality in Surgery , 2015 .

[11]  Sara Condino,et al.  Electromagnetic navigation system for endovascular surgery , 2010 .

[12]  Luis Serra,et al.  DEX‐RAY: AUGMENTED REALITY NEUROSURGICAL NAVIGATION WITH A HANDHELD VIDEO PROBE , 2009, Neurosurgery.

[13]  Fabrizio Cutolo,et al.  Human-PnP: Ergonomic AR Interaction Paradigm for Manual Placement of Rigid Bodies , 2015, AE-CAI.

[14]  Davide Caramella,et al.  Automatic carotid centerline extraction from three-dimensional ultrasound Doppler images , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  Eliot Winer,et al.  Evaluating the Microsoft HoloLens through an augmented reality assembly application , 2017, Defense + Security.

[16]  M Hashizume,et al.  Preliminary Study on the Clinical Application of Augmented Reality Neuronavigation , 2013, Journal of Neurological Surgery—Part A.

[17]  Eigil Samset,et al.  Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances , 2010 .

[18]  D. Louis Collins,et al.  An Evaluation of Depth Enhancing Perceptual Cues for Vascular Volume Visualization in Neurosurgery , 2014, IEEE Transactions on Visualization and Computer Graphics.

[19]  Fabrizio Cutolo,et al.  [POSTER] Hybrid Video/Optical See-Through HMD , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[20]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[21]  J. Marescaux,et al.  Augmented reality in laparoscopic surgical oncology. , 2011, Surgical oncology.

[22]  Leila Besharati Tabrizi,et al.  Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. , 2015, Journal of neurosurgery.

[23]  V Ferrari,et al.  Simultaneous tracking of catheters and guidewires: comparison to standard fluoroscopic guidance for arterial cannulation. , 2014, European journal of vascular and endovascular surgery : the official journal of the European Society for Vascular Surgery.

[24]  U. Thomale,et al.  Augmented reality in intraventricular neuroendoscopy , 2017, Acta Neurochirurgica.

[25]  Mauro Ferrari,et al.  New training methods based on mixed reality for interventional ultrasound: Design and validation , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  Guoyan Zheng,et al.  Computer-Assisted Orthopedic Surgery: Current State and Future Perspective , 2015, Front. Surg..

[27]  M. Citardi,et al.  Augmented reality for endoscopic sinus surgery with surgical navigation: a cadaver study , 2016, International forum of allergy & rhinology.

[28]  Sara Condino,et al.  Electromagnetic navigation platform for endovascular surgery: how to develop sensorized catheters and guidewires , 2012, The international journal of medical robotics + computer assisted surgery : MRCAS.

[29]  Nassir Navab,et al.  Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[30]  K. Schaller,et al.  Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations , 2014, Acta Neurochirurgica.

[31]  H. Iseki,et al.  Endoscopic Augmented Reality Navigation System for Endonasal Transsphenoidal Surgery to Treat Pituitary Tumors: Technical Note , 2002, Neurosurgery.

[32]  Alberto Bianchi,et al.  Navigation in Orthognathic Surgery: 3D Accuracy , 2015, Facial Plastic Surgery.

[33]  R A Kockro,et al.  Minimally invasive cerebral cavernoma surgery using keyhole approaches - solutions for technique-related limitations. , 2009, Minimally invasive neurosurgery : MIN.

[34]  Fabrizio Cutolo,et al.  Augmented reality system for freehand guide of magnetic endovascular devices , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[35]  D. Caramella,et al.  Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. , 2014, Journal of cranio-maxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery.

[36]  A. Meola,et al.  A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom , 2017, Computer assisted surgery.

[37]  Mauro Ferrari,et al.  Value of multidetector computed tomography image segmentation for preoperative planning in general surgery , 2011, Surgical Endoscopy.

[38]  Mauro Ferrari,et al.  Hybrid simulation using mixed reality for interventional ultrasound imaging training , 2015, International Journal of Computer Assisted Radiology and Surgery.

[39]  David J. Hawkes,et al.  Design and evaluation of a system for microscope-assisted guided interventions (MAGI) , 2000 .

[40]  Fabrizio Cutolo,et al.  Augmented reality visualization of deformable tubular structures for surgical simulation , 2016, The international journal of medical robotics + computer assisted surgery : MRCAS.

[41]  Matthias Schulz,et al.  Navigated endoscopic surgery for multiloculated hydrocephalus in children. , 2010, Journal of neurosurgery. Pediatrics.

[42]  O. Ratib,et al.  Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion , 2011, Journal of hepato-biliary-pancreatic sciences.