Spatially aware mobile interface for 3D visualization and interactive surgery planning

While medical images are fundamental in the surgery planning procedure, the process of analysis of such images slice-by-slice is still tedious and inefficient. In this work we introduce a system for exploration of the internal anatomy structures directly on the surface of the real body using a mobile display device as a window to the interior of the patient's body. The method is based on volume visualization of standard computed tomography datasets and augmented reality for interactive visualization of the generated volume. It supports our liver surgery planner method in the analysis of the segmented liver and in the color classification of the vessels. We present a set of experiments showing the system's ability to operate on mobile devices. Quantitative performance results are detailed, and applications in teaching anatomy and doctor-patient communication are discussed.

[1]  Robert A. Greenes,et al.  Radiology systems , 1990 .

[2]  Klaus Mueller,et al.  A practical evaluation of popular volume rendering algorithms , 2000, VVS '00.

[3]  J. Belghiti.  Anatomie chirurgicale du foie , 2001 .

[4]  Martin Kraus,et al.  High-quality pre-integrated volume rendering using hardware-accelerated pixel shading , 2001, HWWS '01.

[5]  Eric Keppel,et al.  Approximating Complex Surfaces by Triangulation of Contour Lines , 1975, IBM J. Res. Dev..

[6]  Heinz-Otto Peitgen,et al.  Impact of virtual tumor resection and computer-assisted risk analysis on operation planning and intraoperative strategy in major hepatic resection. , 2005, Archives of surgery.

[7]  Anderson Maciel,et al.  Anatomic Hepatectomy Planning through Mobile Display Visualization and Interaction , 2012, MMVR.

[8]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[9]  Fabio Bruno,et al.  Visualization of industrial engineering data in Augmented Reality , 2006, J. Vis..

[10]  Anderson Maciel,et al.  Efficient liver surgery planning in 3D based on functional segment classification and volumetric information , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[11]  Marc Levoy,et al.  Efficient ray tracing of volume data , 1990, TOGS.

[12]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[13]  Lee Westover,et al.  Footprint evaluation for volume rendering , 1990, SIGGRAPH.

[14]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[15]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[16]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[17]  H Delingette,et al.  Virtual reality applied to hepatic surgery simulation: the next revolution. , 1998, Annals of surgery.