Mixed reality ultrasound guidance system: a case study in system development and a cautionary tale

PurposeReal-time ultrasound has become a crucial aspect of several image-guided interventions. One of the main constraints of such an approach is the difficulty in interpretability of the limited field of view of the image, a problem that has recently been addressed using mixed reality, such as augmented reality and augmented virtuality. The growing popularity and maturity of mixed reality has led to a series of informal guidelines to direct development of new systems and to facilitate regulatory approval. However, the goals of mixed reality image guidance systems and the guidelines for their development have not been thoroughly discussed. The purpose of this paper is to identify and critically examine development guidelines in the context of a mixed reality ultrasound guidance system through a case study.MethodsA mixed reality ultrasound guidance system tailored to central line insertions was developed in close collaboration with an expert user. This system outperformed ultrasound-only guidance in a novice user study and has obtained clearance for clinical use in humans. A phantom study with 25 experienced physicians was carried out to compare the performance of the mixed reality ultrasound system against conventional ultrasound-only guidance. Despite the previous promising results, there was no statistically significant difference between the two systems.ResultsGuidelines for developing mixed reality image guidance systems cannot be applied indiscriminately. Each design decision, no matter how well justified, should be the subject of scientific and technical investigation. Iterative and small-scale evaluation can readily unearth issues and previously unknown or implicit system requirements.ConclusionsWe recommend a wary eye in development of mixed reality ultrasound image guidance systems emphasizing small-scale iterative evaluation alongside system development. Ultimately, we recommend that the image-guided intervention community furthers and deepens this discussion into best practices in developing image-guided interventions.

[1]  T. M. Silver,et al.  Intraoperative use of real-time ultrasonography in neurosurgery. , 1982, Journal of neurosurgery.

[2]  T. B. Müller,et al.  Intra-operative 3D ultrasound in neurosurgery , 2006, Acta Neurochirurgica.

[3]  R. Ferrari,et al.  Different Image Integration Modalities to Guide AF Ablation: Impact on Procedural and Fluoroscopy Times , 2011, Pacing and clinical electrophysiology : PACE.

[4]  A. Ignee,et al.  EFSUMB Guidelines on Interventional Ultrasound (INVUS), Part V – EUS-Guided Therapeutic Interventions (short version) , 2015, Ultraschall in der Medizin - European Journal of Ultrasound.

[5]  Fergal McCaffery,et al.  Barriers to Adopting Agile Practices When Developing Medical Device Software , 2012, SPICE.

[6]  Christopher A. Troianos,et al.  Internal Jugular Vein and Carotid Artery Anatomic Relation as Determined by Ultrasonography , 1996, Anesthesiology.

[7]  Terry M. Peters,et al.  Augmented Reality Ultrasound Guidance for Central Line Procedures: Preliminary Results , 2015, AE-CAI.

[8]  Fergal McCaffery,et al.  Software Process Improvement Roadmaps - Using Design Patterns to Aid SME's Developing Medical Device Software in the Implementation of IEC 62304 , 2016, SPICE.

[9]  Michael K Gould,et al.  Preventing complications of central venous catheterization. , 2003, The New England journal of medicine.

[10]  A. Verma,et al.  First human experience with real-time integration of intracardiac echocardiography and 3D electroanatomical imaging to guide right free wall accessory pathway ablation. , 2008, Europace : European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology.

[11]  D. W. F. van Krevelen,et al.  A Survey of Augmented Reality Technologies, Applications and Limitations , 2010, Int. J. Virtual Real..

[12]  A. Mittnacht,et al.  Ultrasound-guided vascular access: a comprehensive review. , 2013, Journal of cardiothoracic and vascular anesthesia.

[13]  B. Uretsky,et al.  Anatomical variations of internal jugular vein location: Impact on central venous access , 1991, Critical care medicine.

[14]  F. Lindseth,et al.  Three-dimensional ultrasound-guided placement of ventricular catheters. , 2014, World neurosurgery.

[15]  Peter Jordan,et al.  Standard IEC 62304 - Medical Device Software - Software Lifecycle Processes , 2007 .

[16]  Terry M. Peters,et al.  Guided ultrasound calibration: where, how, and how many calibration fiducials , 2016, International Journal of Computer Assisted Radiology and Surgery.

[17]  Terry M. Peters,et al.  Image-guided interventions and computer-integrated therapy: Quo vadis? , 2016, Medical Image Anal..

[18]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[19]  L. Cavanna,et al.  Central venous catheter care for the patients with cancer: ultrasound-guided insertion should be strongly recommended for internal jugular vein catheterization. , 2013, Annals of oncology : official journal of the European Society for Medical Oncology.

[20]  C. Moore,et al.  Point-of-care ultrasonography. , 2011, The New England journal of medicine.

[21]  Frank Lindseth,et al.  Ultrasound imaging in neurosurgery: approaches to minimize surgically induced image artefacts for improved resection control , 2013, Acta Neurochirurgica.

[22]  Nassir Navab,et al.  Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[23]  Aaron Fenster,et al.  Mechanically assisted 3D ultrasound guided prostate biopsy system. , 2008, Medical physics.

[24]  Terry M. Peters,et al.  Training for Planning Tumour Resection: Augmented Reality and Human Factors , 2015, IEEE Transactions on Biomedical Engineering.

[25]  Mark C. Paulk,et al.  On Empirical Research Into Scrum , 2009 .

[26]  G. Schmidt,et al.  Ultrasound-guided central venous access: what’s new? , 2015, Intensive Care Medicine.

[27]  Mariela Rivera,et al.  Anatomic considerations for central venous cannulation , 2011, Risk management and healthcare policy.

[28]  Gang Wan,et al.  Robot-assisted 3D-TRUS guided prostate brachytherapy: system integration and validation. , 2004, Medical physics.

[29]  Andrew D. Wiles,et al.  Virtual reality-enhanced ultrasound guidance: A novel technique for intracardiac interventions , 2008, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[30]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[31]  Nassir Navab,et al.  On mixed reality environments for minimally invasive therapy guidance: Systems architecture, successes and challenges in their implementation from laboratory to clinic , 2013, Comput. Medical Imaging Graph..

[32]  Terry M. Peters,et al.  A Navigation Platform for Guidance of Beating Heart Transapical Mitral Valve Repair , 2013, IEEE Transactions on Biomedical Engineering.

[33]  F. Dede,et al.  Real‐time ultrasound guided placement of temporary internal jugular vein catheters: Assessment of technical success and complication rates in nephrology practice , 2012, Nephrology.

[34]  Jocelyne Troccaz,et al.  Prostate Biopsy Assistance System with Gland Deformation Estimation for Enhanced Precision , 2009, MICCAI.

[35]  Ben Blachnitzky,et al.  Augmented reality , 2012, 2012 IEEE Hot Chips 24 Symposium (HCS).

[36]  Michael Gessat,et al.  Value of augmented reality-enhanced transesophageal echocardiography (TEE) for determining optimal annuloplasty ring size during mitral valve repair. , 2008, The Annals of thoracic surgery.

[37]  A. Bowdle,et al.  Vascular complications of central venous catheter placement: evidence-based methods for prevention and treatment. , 2014, Journal of cardiothoracic and vascular anesthesia.

[38]  Fergal McCaffery,et al.  Balancing Agility and Discipline in a Medical Device Software Organisation , 2013, SPICE.

[39]  James D. Thomas,et al.  3D echocardiography: a review of the current status and future directions. , 2007, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[40]  Michael Blaivas,et al.  An unseen danger: Frequency of posterior vessel wall penetration by needles during attempts to place internal jugular vein central catheters using ultrasound guidance* , 2009, Critical care medicine.

[41]  Catherine M. Otto,et al.  Practice of Clinical Echocardiography , 2007 .

[42]  T. Peters,et al.  Poly(vinyl alcohol) cryogel phantoms for use in ultrasound and MR imaging , 2004, Physics in medicine and biology.

[43]  Victor Rodrigues,et al.  Agile Development in a Medical Device Company , 2008, Agile 2008 Conference.

[44]  S. Price,et al.  Central venous cannulation , 2014 .

[45]  Ziv Yaniv,et al.  Applications of Augmented Reality in the Operating Room , 2015 .

[46]  Fergal McCaffery,et al.  Creation of an IEC 62304 compliant software development plan , 2016, J. Softw. Evol. Process..