Depth image-based representation and compression for static and animated 3-D objects

This paper describes a new family of three-dimensional (3-D) representations for computer graphics and animation, called depth image-based representations (DIBR), which have been adopted into MPEG-4 Part16: Animation Framework eXtension (AFX). Idea of the approach is to build a compact and photorealistic representation of a 3-D object or scene without using polygonal mesh. Instead, images accompanied by depth values for each pixel are used. This type of representation allows us to build and render novel views of objects and scene with an interactive rate. There are many different methods for the image-based rendering with depths, and the DIBR format is designed to efficiently represent the information necessary for such methods. The main formats of the DIBR family are SimpleTexture (an image together with depth array), PointTexture (an image with multiple pixels along each line of sight), and OctreeImage (octree-like data structure together with a set of images containing viewport parameters). In order to store and transmit the DIBR object, we develop a compression algorithm and bitstream format for OctreeImage representation.

[1]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[2]  Alexander Zhirkov.  Binary Volumetric Octree Representation for Image-Based Rendering , 2001 .

[3]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.

[4]  Glen G. Langdon,et al.  Universal modeling and coding , 1981, IEEE Trans. Inf. Theory.

[5]  Marc Levoy,et al.  The Use of Points as a Display Primitive , 2000 .

[6]  Dani Lischinski,et al.  Image-Based Rendering for Non-Diffuse Synthetic Scenes , 1998, Rendering Techniques.

[7]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[8]  Leonard McMillan,et al.  A List-Priority Rendering Algorithm for Redisplaying Projected Surfaces , 1995 .

[9]  Anselmo Lastra,et al.  LDI tree: a hierarchical representation for image-based rendering , 1999, SIGGRAPH.

[10]  G. Nigel Martin,et al.  * Range encoding: an algorithm for removing redundancy from a digitised message , 1979 .

[11]  Sang Uk Lee,et al.  Compact encoding of 3-D voxel surfaces based on pattern code representation , 2002, IEEE Trans. Image Process..

[12]  Ian H. Witten,et al.  Data Compression Using Adaptive Coding and Partial String Matching , 1984, IEEE Trans. Commun..

[13]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[14]  David Salesin,et al.  Hierarchical image caching for accelerated walkthroughs of complex environments , 1996, SIGGRAPH.

[15]  Chris Bregler.  Video Based Animation Techniques for Human Motion , 1999 .

[16]  Manuel Menezes de Oliveira Neto,et al.  Relief texture mapping , 2000, SIGGRAPH.

[17]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[18]  Wolfgang Stuerzlinger,et al.  A Three Dimensional Image Cache for Virtual Reality , 1996, Comput. Graph. Forum.

[19]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[20]  Manuel Menezes de Oliveira Neto,et al.  Image-based objects , 1999, SI3D.

[21]  P. Debevec,et al.  Image-based modeling, rendering, and lighting , 2002, IEEE Computer Graphics and Applications.

[22]  Matthias Zwicker,et al.  Surfels: surface elements as rendering primitives , 2000, SIGGRAPH.

[23]  H. Coxeter,et al.  Generators and relations for discrete groups , 1957 .

[24]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.