Augmented reality in robotic assisted orthopaedic surgery: A pilot study

BACKGROUND The research and development of augmented-reality (AR) technologies in surgical applications has seen an evolution of the traditional user-interfaces (UI) utilised by clinicians when conducting robot-assisted orthopaedic surgeries. The typical UI for such systems relies on surgeons managing 3D medical imaging data in the 2D space of a touchscreen monitor, located away from the operating site. Conversely, AR can provide a composite view overlaying the real surgical scene with co-located virtual holographic representations of medical data, leading to a more immersive and intuitive operator experience. MATERIALS AND METHODS This work explores the integration of AR within an orthopaedic setting by capturing and replicating the UI of an existing surgical robot within an AR head-mounted display worn by the clinician. The resulting mixed-reality workflow enabled users to simultaneously view the operating-site and real-time holographic operating informatics when carrying out a robot-assisted patellofemoral-arthroplasty (PFA). Ten surgeons were recruited to test the impact of the AR system on procedure completion time and operating surface roughness. RESULTS & DISCUSSION The integration of AR did not appear to require subjects to significantly alter their surgical techniques, which was demonstrated by non-significant changes to the study's clinical metrics, with a statistically insignificant mean increase in operating time (+0.778 s, p = 0.488) and a statistically insignificant change in mean surface roughness (p = 0.274). Additionally, a post-operative survey indicated a positive consensus on the usability of the AR system without incurring noticeable physical distress such as eyestrain or fatigue. CONCLUSIONS Overall, these study results demonstrated a successful integration of AR technologies within the framework of an existing robot-assisted surgical platform with no significant negative effects in two quantitative metrics of surgical performance, and a positive outcome relating to user-centric and ergonomic evaluation criteria.

[1]  Leonid L. Chepelev,et al.  Applying Modern Virtual and Augmented Reality Technologies to Medical Images and Models , 2018, Journal of Digital Imaging.

[2]  J. Hsu,et al.  Virtual Reality and Augmented Reality—Translating Surgical Training into Surgical Technique , 2020, Current Reviews in Musculoskeletal Medicine.

[3]  J. J. Jakimowicz,et al.  Suturing training in Augmented Reality: gaining proficiency in suturing skills faster , 2008, Surgical Endoscopy.

[4]  Nicholas A. Giovinco,et al.  A Heads-Up Display for Diabetic Limb Salvage Surgery , 2014, Journal of diabetes science and technology.

[5]  M. Conditt,et al.  Robotic Arm-assisted UKA Improves Tibial Component Alignment: A Pilot Study , 2010, Clinical orthopaedics and related research.

[6]  B. Brenseke,et al.  Evaluating the Use of Mixed Reality to Teach Gross and Microscopic Respiratory Anatomy , 2020, Medical Science Educator.

[7]  Frederic Picard,et al.  High Degree of Accuracy of a Novel Image-free Handheld Robot for Unicondylar Knee Arthroplasty in a Cadaveric Study , 2015, Clinical orthopaedics and related research.

[8]  Marina Yiasemidou,et al.  Role of immersive technologies in healthcare education during the COVID-19 epidemic , 2020, Scottish medical journal.

[9]  Raju Vaishya,et al.  Bone cement. , 2013, Journal of clinical orthopaedics and trauma.

[10]  Xiangyu Wang,et al.  A study on the benefits of augmented reality in retaining working memory in assembly tasks: A focus on differences in gender , 2013 .

[11]  Matthew Christopher Davis,et al.  Virtual Interactive Presence in Global Surgical Education: International Collaboration Through Augmented Reality. , 2016, World neurosurgery.

[12]  Ekkehard Euler,et al.  Hybrid Navigation Interface for Orthopedic and Trauma Surgery , 2006, MICCAI.

[13]  Antonija Mitrovic,et al.  Intelligent Augmented Reality Training for Motherboard Assembly , 2015, International Journal of Artificial Intelligence in Education.

[14]  K. Schaller,et al.  Augmented reality-assisted bypass surgery: embracing minimal invasiveness. , 2015, World neurosurgery.

[15]  N. Navab,et al.  Advanced Medical Displays: A Literature Review of Augmented Reality , 2008, Journal of Display Technology.

[16]  B. Jaramaz,et al.  Accuracy assessment of a novel image-free handheld robot for Total Knee Arthroplasty in a cadaveric study , 2018, Computer assisted surgery.

[17]  D. Louis Collins,et al.  Augmented reality in neurovascular surgery: feasibility and first uses in the operating room , 2015, International Journal of Computer Assisted Radiology and Surgery.

[18]  Na Chen,et al.  Modeling and simulation of an anatomy teaching system , 2019, Vis. Comput. Ind. Biomed. Art.

[19]  Fabrizio Cutolo.  Letter to the Editor on “Augmented Reality Based Navigation for Computer Assisted Hip Resurfacing: A Proof of Concept Study” , 2019, Annals of Biomedical Engineering.

[20]  Adrien Bartoli,et al.  Use of augmented reality in laparoscopic gynecology to visualize myomas. , 2017, Fertility and sterility.

[21]  Ramin Javan,et al.  Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography , 2018, International Journal of Computer Assisted Radiology and Surgery.

[22]  S. Tsuda,et al.  Augmented reality telementoring (ART) platform: a randomized controlled trial to assess the efficacy of a new surgical education technology , 2014, Surgical Endoscopy.

[23]  L. Rimondini,et al.  In Vitro Behaviour of Osteoblasts Cultured on Orthopaedic Biomaterials with Different Surface Roughness, Uncoated and Fluorohydroxyapatite-Coated, Relative to the in Vivo Osteointegration Rate , 2003, The International journal of artificial organs.

[24]  Brice Gayet,et al.  Augmented Reality Navigation Surgery Facilitates Laparoscopic Rescue of Failed Portal Vein Embolization. , 2016, Journal of the American College of Surgeons.