Multi-Touch Table System for Medical Visualization: Application to Orthopedic Surgery Planning

Medical imaging plays a central role in a vast range of healthcare practices. The usefulness of 3D visualizations has been demonstrated for many types of treatment planning. Nevertheless, full access to 3D renderings outside of the radiology department is still scarce even for many image-centric specialties. Our work stems from the hypothesis that this under-utilization is partly due to existing visualization systems not taking the prerequisites of this application domain fully into account. We have developed a medical visualization table intended to better fit the clinical reality. The overall design goals were two-fold: similarity to a real physical situation and a very low learning threshold. This paper describes the development of the visualization table with focus on key design decisions. The developed features include two novel interaction components for touch tables. A user study including five orthopedic surgeons demonstrates that the system is appropriate and useful for this application domain.

[1]  N. Navab,et al.  Advanced Medical Displays: A Literature Review of Augmented Reality , 2008, Journal of Display Technology.

[2]  Tony DeRose,et al.  Determining the benefits of direct-touch, bimanual, and multifinger input on a multitouch workstation , 2009, Graphics Interface.

[3]  Andreas Schilling,et al.  The DabR - A multitouch system for intuitive 3D scene navigation , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[4]  Katherine P Andriole,et al.  Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day. , 2011, Radiology.

[5]  M. Sheelagh T. Carpendale,et al.  Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques , 2007, CHI.

[6]  Henning Scharsach.  Advanced GPU Raycasting , 2005 .

[7]  Philip L. Davidson,et al.  A screen-space formulation for 2D and 3D direct manipulation , 2009, UIST '09.

[8]  Dimitre Novatchev,et al.  Chunking and Phrasing and the Design of Human-Computer Dialogues - Response , 1986, IFIP Congress.

[9]  Chris North,et al.  Understanding Multi-touch Manipulation for Surface Computing , 2009, INTERACT.

[10]  Michitaka Hirose,et al.  3D User Interfaces: New Directions and Perspectives , 2008, IEEE Computer Graphics and Applications.

[11]  Timo Engelke,et al.  Adapting X3D for multi-touch environments , 2008, Web3D '08.

[12]  Laurent Grisoni,et al.  3D positioning techniques for multi-touch displays , 2009, VRST '09.

[13]  M. Sheelagh T. Carpendale,et al.  Evaluating Information Visualizations , 2008, Information Visualization.

[14]  Örjan Smedby,et al.  Advanced 3D visualization in student-centred medical education , 2008, Medical teacher.

[15]  Giuseppe De Pietro,et al.  A user interface for VR-ready 3D medical imaging by off-the-shelf input devices , 2010, Comput. Biol. Medicine.

[16]  M. Sheelagh T. Carpendale,et al.  Fluid integration of rotation and translation , 2005, CHI.

[17]  M. Thali,et al.  Post-mortem forensic neuroimaging: correlation of MSCT and MRI findings with autopsy results. , 2007, Forensic science international.

[18]  Tobias Isenberg,et al.  Interactive Exploratory Visualization of 2D Vector Fields , 2008, Comput. Graph. Forum.

[19]  Ioannis A. Kakadiaris,et al.  Interaction with Medical Volume Data on a Projection Workbench , 2001 .

[20]  J. McLachlan,et al.  Teaching anatomy without cadavers , 2004, Medical education.

[21]  Junji Okuda,et al.  Preoperative simulation of vascular anatomy by three-dimensional computed tomography imaging in laparoscopic gastric cancer surgery. , 2003, Journal of the American College of Surgeons.

[22]  Naohiro Sata,et al.  Preoperative virtual simulation of adrenal tumors , 2008, Abdominal Imaging.

[23]  Tobias Isenberg,et al.  FI3D: Direct-Touch Interaction for the Exploration of 3D Scientific Visualization Spaces , 2010, IEEE Transactions on Visualization and Computer Graphics.

[24]  Mary Czerwinski,et al.  An exploratory study of co-located collaborative visual analytics around a tabletop display , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[25]  Carl Gutwin,et al.  TNT: improved rotation and translation on digital tables , 2006, Graphics Interface.

[26]  Julian L Burton,et al.  Clinical, educational, and epidemiological value of autopsy , 2007, The Lancet.

[27]  M. Sheelagh T. Carpendale,et al.  Sticky tools: full 6DOF force-based interaction for multi-touch tables , 2009, ITS '09.

[28]  Yvonne Rogers,et al.  Finger talk: collaborative decision-making using talk and fingertip interaction around a tabletop display , 2004, CHI EA '04.

[29]  Anders Ynnerman,et al.  Full Body Virtual Autopsies using a State-of-the-art Volume Rendering Pipeline , 2006, IEEE Transactions on Visualization and Computer Graphics.

[30]  Osman Ratib,et al.  A lung segmentectomy performed with 3D reconstruction images available on the operating table with an iPad. , 2011, Interactive cardiovascular and thoracic surgery.

[31]  Yvonne Rogers,et al.  Collaboration and interference: awareness with mice or touch input , 2008, CSCW.

[32]  Fotis Sotiropoulos,et al.  Slice WIM: a multi-surface, multi-touch interface for overview+detail exploration of volume datasets in virtual reality , 2011, SI3D.

[33]  P. Hasleton,et al.  Discrepancies between clinical and autopsy diagnosis and the value of post mortem histology; a meta‐analysis and review , 2005, Histopathology.

[34]  Bernd Fröhlich,et al.  On 3D input devices , 2006, IEEE Computer Graphics and Applications.

[35]  Laurent Grisoni,et al.  The effect of DOF separation in 3D manipulation tasks with multi-touch displays , 2010, VRST '10.

[36]  M. Sheelagh T. Carpendale,et al.  Rotation and translation mechanisms for tabletop interaction , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[37]  R. Malthaner,et al.  The feasibility of three-dimensional displays of the thorax for preoperative planning in the surgical treatment of lung cancer. , 2007, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[38]  Bernhard Preim,et al.  Usability Comparison of Mouse-Based Interaction Techniques for Predictable 3d Rotation , 2005, Smart Graphics.

[39]  Bernd Fröhlich,et al.  The Responsive Workbench [virtual work environment] , 1994, IEEE Computer Graphics and Applications.

[40]  Nicola J. Robertson,et al.  Post-mortem MR and CT imaging in fetuses, newborns and children: an evidenced based approach , 2010 .