On the Use of Virtual Reality for Medical Imaging Visualization

Advanced visualization of medical imaging has been a motive for research due to its value for disease analysis, surgical planning, and academical training. More recently, attention has been turning toward mixed reality as a means to deliver more interactive and realistic medical experiences. However, there are still many limitations to the use of virtual reality for specific scenarios. Our intent is to study the current usage of this technology and assess the potential of related development tools for clinical contexts. This paper focuses on virtual reality as an alternative to today’s majority of slice-based medical analysis workstations, bringing more immersive three-dimensional experiences that could help in cross-slice analysis. We determine the key features a virtual reality software should support and present today’s software tools and frameworks for researchers that intend to work on immersive medical imaging visualization. Such solutions are assessed to understand their ability to address existing challenges of the field. It was understood that most development frameworks rely on well-established toolkits specialized for healthcare and standard data formats such as DICOM. Also, game engines prove to be adequate means of combining software modules for improved results. Virtual reality seems to remain a promising technology for medical analysis but has not yet achieved its true potential. Our results suggest that prerequisites such as real-time performance and minimum latency pose the greatest limitations for clinical adoption and need to be addressed. There is also a need for further research comparing mixed realities and currently used technologies.

[1]  Christian Hansen,et al.  Highly immersive virtual reality laparoscopy simulation: development and future aspects , 2018, International Journal of Computer Assisted Radiology and Surgery.

[2]  Ronei Marcos de Moraes,et al.  SimCEC: A Collaborative VR-Based Simulator for Surgical Teamwork Education , 2018, Comput. Entertain..

[3]  Ioana-Raluca Adochiei,et al.  A WEB Platform for Rendring and Viewing MRI Volumes using Real-Time Raytracing Principles , 2019, 2019 11th International Symposium on Advanced Topics in Electrical Engineering (ATEE).

[4]  Abhishek Kolagunda,et al.  A multiparametric magnetic resonance imaging-based virtual reality surgical navigation tool for robotic-assisted radical prostatectomy. , 2019, Turkish journal of urology.

[5]  Juan Antonio Juanes Méndez,et al.  Virtual reality medical training system , 2016, TEEM.

[6]  Paulo Dias,et al.  Using Virtual Reality to Increase Motivation in Poststroke Rehabilitation , 2019, IEEE Computer Graphics and Applications.

[7]  Yonghong Peng,et al.  A Novel WebVR-Based Lightweight Framework for Virtual Visualization of Blood Vasculum , 2018, IEEE Access.

[8]  Gabriel Cristóbal,et al.  Midbrain volume segmentation using active shape models and LBPs , 2013, Optics & Photonics - Optical Engineering + Applications.

[9]  A. Kesselman,et al.  A Role for Virtual Reality in Planning Endovascular Procedures. , 2018, Journal of vascular and interventional radiology : JVIR.

[10]  Olaf Dössel,et al.  An interactive virtual reality environment for analysis of clinical atrial arrhythmias and ablation planning , 2017, 2017 Computing in Cardiology (CinC).

[11]  Dieter Schmalstieg,et al.  Multi-threaded integration of HTC-Vive and MeVisLab , 2018, Medical Imaging.

[12]  Y. Laufer,et al.  Virtual reality therapy in paediatric rehabilitation , 2011 .

[13]  Gorjan Alagic,et al.  #p , 2019, Quantum information & computation.

[14]  Nicolas Toussaint,et al.  Virtual interaction and visualisation of 3D medical imaging data with VTK and Unity , 2018, Healthcare technology letters.

[15]  Nigel W. John,et al.  VR Cardiovascular Blood Simulation as Decision Support for the Future Cyber Hospital , 2017, 2017 International Conference on Cyberworlds (CW).

[16]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[17]  Pramodita Sharma 2012 , 2013, Les 25 ans de l’OMC: Une rétrospective en photos.

[18]  O. Giotakos,et al.  [Virtual reality therapy in anxiety disorders]. , 2016, Psychiatrike = Psychiatriki.

[19]  Bernhard Preim,et al.  Reusable Visualizations and Animations for Surgery Planning , 2010, Comput. Graph. Forum.

[20]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[21]  Aashish Chaudhary,et al.  Enhancements to VTK enabling scientific visualization in immersive environments , 2017, 2017 IEEE Virtual Reality (VR).

[22]  Paul Zikas,et al.  Transforming medical education and training with VR using M.A.G.E.S. , 2018, SIGGRAPH ASIA Posters.

[23]  Xin Wang,et al.  Virtual Reality Training System for Surgical Anatomy , 2018, AIVR 2018.

[24]  A. Ayoub,et al.  The application of virtual reality and augmented reality in Oral & Maxillofacial Surgery , 2019, BMC Oral Health.

[25]  Wenjun Tan,et al.  Computer assisted system for precise lung surgery based on medical image computing and mixed reality , 2018, Health Information Science and Systems.

[26]  Jason Smith,et al.  VRvisu: A Tool for Virtual Reality Based Visualization of Medical Data , 2017, 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[27]  Sébastien Ourselin,et al.  A mixed-reality surgical trainer with comprehensive sensing for fetal laser minimally invasive surgery , 2018, International Journal of Computer Assisted Radiology and Surgery.

[28]  Jun Peng,et al.  Real-Time Needle Force Modeling for VR-Based Renal Biopsy Training with Respiratory Motion Using Direct Clinical Data , 2019, Applied bionics and biomechanics.

[29]  Bennett P. Samuel,et al.  Transcatheter Intervention for Paravalvular Leak in Mitroflow Bioprosthetic Pulmonary Valve , 2019, Journal of Structural Heart Disease.

[30]  Jorge E. Valdez-García,et al.  Assessing the Effectiveness of Teaching Anatomy with Virtual Reality , 2019, ICETC.

[31]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[32]  Karen J. Reynolds,et al.  Virtual reality for medical training: the state-of-the-art , 2015, J. Simulation.

[33]  Francisco José García-Peñalvo,et al.  NextMed, Augmented and Virtual Reality platform for 3D medical imaging visualization: Explanation of the software platform developed for 3D models visualization related with medical images using Augmented and Virtual Reality technology , 2019, TEEM.

[34]  Song Tian,et al.  Comparison of medical image 3D reconstruction rendering methods for robot-assisted surgery , 2017, 2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM).

[35]  Ziv Yaniv,et al.  The Image-Guided Surgery Toolkit IGSTK: An Open Source C++ Software Toolkit , 2007, Journal of Digital Imaging.

[36]  E. F. Manffra,et al.  The Use of Virtual Reality Rehabilitation for Individuals Post Stroke , 2019 .

[37]  Arthur W. Toga,et al.  NIVR: Neuro imaging in virtual reality , 2017, 2017 IEEE Virtual Reality (VR).

[38]  Paulo Amorim,et al.  InVesalius: An Interactive Rendering Framework for Health Care Support , 2015, ISVC.

[39]  Samir Benbelkacem,et al.  An Interactive VR System for Anatomy Training , 2020, Mixed Reality and Three-Dimensional Computer Graphics.

[40]  Slim Abdennadher,et al.  PathoGenius VR: VR medical training , 2019, PerDis.

[41]  Long Chen,et al.  Recent Developments and Future Challenges in Medical Mixed Reality , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[42]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[43]  Steven C. Horii,et al.  Review: Understanding and Using DICOM, the Data Interchange Standard for Biomedical Imaging , 1997, J. Am. Medical Informatics Assoc..

[44]  Brenda K Wiederhold Are We Ready for Online Virtual Reality Therapy? , 2018, Cyberpsychology Behav. Soc. Netw..

[45]  R A Robb,et al.  Analyze: a comprehensive, operator-interactive software package for multidimensional medical image display and analysis. , 1989, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[46]  Stefanie Speidel,et al.  IMHOTEP: virtual reality framework for surgical applications , 2018, International Journal of Computer Assisted Radiology and Surgery.

[47]  Charles E Kahn,et al.  DICOM and radiology: past, present, and future. , 2007, Journal of the American College of Radiology : JACR.