Comparison of Augmented Reality Display Techniques to Support Medical Needle Insertion

Augmented reality (AR) may be a useful technique to overcome issues of conventionally used navigation systems supporting medical needle insertions, like increased mental workload and complicated hand-eye coordination. Previous research primarily focused on the development of AR navigation systems designed for specific displaying devices, but differences between employed methods have not been investigated before. To this end, a user study involving a needle insertion task was conducted comparing different AR display techniques with a monitor-based approach as baseline condition for the visualization of navigation information. A video see-through stationary display, an optical see-through head-mounted display and a spatial AR projector-camera-system were investigated in this comparison. Results suggest advantages of using projected navigation information in terms of lower task completion time, lower angular deviation and affirmative subjective participant feedback. Techniques requiring the intermediate view on screens, i.e. the stationary display and the baseline condition, showed less favorable results. Thus, benefits of providing AR navigation information compared to a conventionally used method could be identified. Significant objective measures results, as well as an identification of advantages and disadvantages of individual display techniques contribute to the development and design of improved needle navigation systems.

[1]  Frank Sauer,et al.  An augmented reality system for MR image-guided needle biopsy: initial results in a swine model. , 2006, Radiology.

[2]  J. Debus,et al.  Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. , 2008, International journal of radiation oncology, biology, physics.

[3]  Kai Lawonn,et al.  Comparison of Projective Augmented Reality Concepts to Support Medical Needle Insertion , 2019, IEEE Transactions on Visualization and Computer Graphics.

[4]  Peter Kazanzides,et al.  Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display , 2017, International Journal of Computer Assisted Radiology and Surgery.

[5]  Kosuke Sato,et al.  Shadowless Projector: Suppressing Shadows in Projection Mapping with Micro Mirror Array Plate , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[6]  Ron Kikinis,et al.  Augmented reality visualization for CT-guided interventions: system description, feasibility, and initial evaluation in an abdominal phantom. , 2006, Radiology.

[7]  Thomas B. Moeslund,et al.  Augmented Reality Technology for Displaying Close-Proximity Sub-Surface Positions , 2019, INTERACT.

[8]  Christian Hansen,et al.  Projector‐based augmented reality system for interventional visualization inside MRI scanners , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[9]  Nassir Navab,et al.  Auditory and Visio-Temporal Distance Coding for 3-Dimensional Perception in Medical Augmented Reality , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.

[10]  Gudrun Klinker,et al.  Semi-Parametric Color Reproduction Method for Optical See-Through Head-Mounted Displays , 2015, IEEE Transactions on Visualization and Computer Graphics.

[11]  Ron Kikinis,et al.  Instrument-mounted displays for reducing cognitive load during surgical navigation , 2017, International Journal of Computer Assisted Radiology and Surgery.

[12]  Ron Kikinis,et al.  Auditory feedback to support image-guided medical needle placement , 2017, International Journal of Computer Assisted Radiology and Surgery.

[13]  Gabor Fichtinger,et al.  Augmented reality visualization with image overlay for MRI-guided intervention: accuracy for lumbar spinal procedures with a 1.5-T MRI system. , 2012, AJR. American journal of roentgenology.

[14]  Holger Regenbrecht,et al.  Real-Time Radiometric Compensation for Optical See-Through Head-Mounted Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[15]  G. Strauß,et al.  Image‐guided navigation: the surgeon's perspective on performance consequences and human factors issues , 2009, The international journal of medical robotics + computer assisted surgery : MRCAS.

[16]  C. Pfirrmann,et al.  Augmented Reality–Guided Lumbar Facet Joint Injections , 2018, Investigative radiology.

[17]  L. Faulkner Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[18]  Hartwig R. Siebner,et al.  Rotated alphanumeric characters do not automatically activate frontoparietal areas subserving mental rotation , 2009, NeuroImage.

[19]  Florian Heinrich,et al.  HoloInjection: augmented reality support for CT-guided spinal needle injections , 2019, Healthcare technology letters.

[20]  Bingyao Huang,et al.  End-To-End Projector Photometric Compensation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  HerrlichMarc,et al.  Put That Needle There: Customized Flexible On-Body Thin-Film Displays for Medical Navigation , 2020 .

[22]  Dietrich Manzey,et al.  Impact of image-guided surgery on surgeons' performance: a literature review , 2016 .

[23]  R. Shepard,et al.  The time required to prepare for a rotated stimulus , 1973, Memory & cognition.

[24]  Martin Deli,et al.  A novel Laser Navigation System reduces radiation exposure and improves accuracy and workflow of CT-guided spinal interventions: a prospective, randomized, controlled, clinical trial in comparison to conventional freehand puncture. , 2013, European journal of radiology.

[25]  Adam Rankin,et al.  Hologram stability evaluation for Microsoft HoloLens , 2017, Medical Imaging.

[26]  Guy Brock,et al.  Experience With More Than 500 Minimally Invasive Hepatic Procedures , 2008, Annals of surgery.

[27]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[28]  Lutz-Peter Nolte,et al.  A navigation system for percutaneous needle interventions based on PET/CT images: Design, workflow and error analysis of soft tissue and bone punctures , 2011, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[29]  Manuela Chessa,et al.  Perception and Action in Peripersonal Space: A Comparison Between Video and Optical See-Through Augmented Reality Devices , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[30]  Maria del Carmen Juan Lizandra,et al.  An Augmented Reality System for the Treatment of Phobia to Small Animals Viewed Via an Optical See-Through HMD: Comparison With a Similar System Viewed Via a Video See-Through HMD , 2011, Int. J. Hum. Comput. Interact..

[31]  Ernesto Damiani,et al.  Augmented reality technologies, systems and applications , 2010, Multimedia Tools and Applications.

[32]  Gerlig Widmann,et al.  Accuracy and diagnostic yield of CT-guided stereotactic liver biopsy of primary and secondary liver tumors , 2011, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[33]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[34]  Arianeb Mehrabi,et al.  Towards markerless navigation for percutaneous needle insertions , 2015, International Journal of Computer Assisted Radiology and Surgery.

[35]  Andrew Y. C. Nee,et al.  Augmented reality applications in design and manufacturing , 2012 .

[36]  Ramin Javan,et al.  Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography , 2018, International Journal of Computer Assisted Radiology and Surgery.

[37]  Henk G. Stassen,et al.  Theoretical background and conceptual solution for depth perception and eye-hand coordination problems in laparoscopic surgery , 1999 .

[38]  Sameer Chopra,et al.  Three‐dimensional navigation system integrating position‐tracking technology with a movable tablet display for percutaneous targeting , 2015, BJU international.

[39]  Anselm Grundhöfer,et al.  Robust, Error-Tolerant Photometric Projector Compensation , 2015, IEEE Transactions on Image Processing.

[40]  Steven K. Feiner,et al.  Perceptual issues in augmented reality revisited , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[41]  Tamara Munzner,et al.  Visualization Analysis and Design , 2014, A.K. Peters visualization series.

[42]  Stefan Weber,et al.  Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies , 2012, International Journal of Computer Assisted Radiology and Surgery.

[43]  Mark R. Cutkosky,et al.  HoloNeedle: Augmented Reality Guidance System for Needle Placement Investigating the Advantages of Three-Dimensional Needle Shape Reconstruction , 2018, IEEE Robotics and Automation Letters.

[44]  Jan Fritz,et al.  Freehand real-time MRI-guided lumbar spinal injection procedures at 1.5 T: feasibility, accuracy, and safety. , 2009, AJR. American journal of roentgenology.

[45]  Bradford J. Wood,et al.  Smartphone Augmented Reality CT-Based Platform for Needle Insertion Guidance: A Phantom Study , 2020, CardioVascular and Interventional Radiology.

[46]  Guillaume Moreau,et al.  A new typology of augmented reality applications , 2012, AH '12.