View Interpolation for Medical Images on Autostereoscopic Displays

We present an approach for efficient rendering and transmitting views to a high-resolution autostereoscopic display for medical purposes. Displaying biomedical images on an autostereoscopic display poses different requirements than in a consumer case. For medical usage, it is essential that the perceived image represents the actual clinical data and offers sufficiently high quality for diagnosis or understanding. Autostereoscopic display of multiple views introduces two hurdles: transmission of multi-view data through a bandwidth-limited channel and the computation time of the volume rendering algorithm. We address both issues by generating and transmitting limited set of views enhanced with a depth signal per view. We propose an efficient view interpolation and rendering algorithm at the receiver side based on texture+depth data representation, which can operate with a limited amount of views. We study the main artifacts that occur during rendering-occlusions, and we quantify them first for a synthetic model and then for real-world biomedical data. The experimental results allow us to quantify the peak signal-to-noise ratio for rendered texture and depth as well as the amount of disoccluded pixels as a function of the angle between surrounding cameras.

[1]  N. A. Dodgson Autostereo displays: 3D without glasses , 1997 .

[2]  Timo Ropinski,et al.  Accelerating Volume Raycasting using Occlusion Frustums , 2008, VG/PBG@SIGGRAPH.

[3]  Cees van Berkel,et al.  Image preparation for 3D LCD , 1999 .

[4]  Daniel Ruijters Integrating autostereoscopic multi-view lenticular displays in minimally invasive angiography , 2008 .

[5]  Carlos Vázquez,et al.  3D-TV: coding of disocclusions for 2D+depth representation of multi-view images , 2008 .

[6]  Marc Levoy,et al.  Efficient ray tracing of volume data , 1990, TOGS.

[7]  Jr. Leonard McMillan,et al.  An Image-Based Approach to Three-Dimensional Computer Graphics , 1997 .

[8]  Janusz Konrad,et al.  Geometry-based estimation of occlusions from video frame pairs , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[9]  Yannick Morvan,et al.  Acquisition, compression and rendering of depth and texture for multi-view video , 2009 .

[10]  Andreas Abildgaard,et al.  An autostereoscopic 3D display can improve visualization of 3D models from intracranial MR angiography , 2010, International Journal of Computer Assisted Radiology and Surgery.

[11]  Tzong-Jer Chen,et al.  Quality of Compressed Medical Images , 2007, Journal of Digital Imaging.

[12]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[13]  Toshiaki Fujii,et al.  View generation with 3D warping using depth information for FTV , 2009, Signal Process. Image Commun..

[14]  Hongen Liao,et al.  3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay , 2010, IEEE Transactions on Biomedical Engineering.

[15]  Erwin Keeve,et al.  Stereoscopic augmented reality system for computer-assisted surgery , 2001, CARS.

[16]  Peter H. N. de With,et al.  Quality improving techniques in DIBR for free-viewpoint video , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[17]  Peter H. N. de With,et al.  System architecture for free-viewpoint video and 3D-TV , 2008, IEEE Transactions on Consumer Electronics.

[18]  Masayuki Tanimoto FTV (Free viewpoint TV) and Creation of Ray-Based Image Engineering , 2009 .

[19]  Peter Stoeter,et al.  Virtual reality system for planning minimally invasive neurosurgery. Technical note. , 2008, Journal of neurosurgery.

[20]  Peter H. N. de With,et al.  Free-viewpoint depth image based rendering , 2010, J. Vis. Commun. Image Represent..

[21]  J F Walkup,et al.  Image compression in signal-dependent noise. , 1999, Applied optics.

[22]  Peter H. N. de With,et al.  Quality improving techniques for free-viewpoint DIBR , 2010, Electronic Imaging.

[23]  Thomas Hübner,et al.  SINGLE-PASS MULTIVIEW VOLUME RENDERING , 2007 .

[24]  Ken Masamune,et al.  Fusion of Laser Guidance and 3-D Autostereoscopic Image Overlay for Precision-Guided Surgery , 2008, MIAR.

[25]  Svitlana Zinger,et al.  iGLANCE: Transmission to medical high definition autostereoscopic displays , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[26]  Daniel Ruijters Dynamic Resolution in GPU-Accelerated Volume Rendering to Autostereoscopic Multiview Lenticular Displays , 2009, EURASIP J. Adv. Signal Process..