GPU-Based 3D Video Object Synthesis and Its Quality Assessment

We present an algorithm and its implementation on a GPU for the view synthesis of 3D video objects. The algorithm is based on the theory of trifocal transfer using dense depth information of two cameras. We present issues that arise when using the pipeline architecture of the GPU for non-computer graphics algorithms and introduce approaches how to solve these problems with focus on our application. In order to assess the quality achieved by the algorithm, we present a method for generating ground truth both for synthetic and real imagery. We present first results of subjective tests for quality evaluation.

[1]  Peter Eisert,et al.  Free viewpoint video extraction, representation, coding, and rendering , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[2]  Reinhard Koch,et al.  Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling , 2004 .

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  C. Weigel,et al.  Advanced 3D Video Object Synthesis Based on Trilinear Tensors , 2006, 2006 IEEE International Symposium on Consumer Electronics.

[5]  Ruigang Yang,et al.  A Unified Approach To Real-Time, Multi-Resolution, Multi-Baseline 2d View Synthesis And 3d Depth Estimation Using Commodity Graphics Hardware , 2004, Int. J. Image Graph..

[6]  Amnon Shashua,et al.  Novel View Synthesis by Cascading Trilinear Tensors , 1998, IEEE Trans. Vis. Comput. Graph..

[7]  T. Wiegand,et al.  EVALUATION OF 3 D RECONSTRCUTION USING MULTIVIEW BACKPROJECTION , 2006 .

[8]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[9]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[10]  Ian D. Reid,et al.  Novel View Specification and Synthesis , 2002, BMVC.

[11]  Ruigang Yang,et al.  A versatile stereo implementation on commodity graphics hardware , 2005, Real Time Imaging.

[12]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[13]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Heiko Hirschm,et al.  Accurate and Efcient Stereo Processing by Semi-Global Matching and Mutual Information , 2005 .

[15]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.