Declipse 2: multi-layer image and depth with transparency made practical

To be able to accommodate different viewing conditions (e.g. different display sizes, display types, viewing distance), a 3D format is required where the depth effect can be adjusted. Within MPEG, the image-and-depth format has been standardized. This format is bandwidth efficient, highly flexible and therefore well suited to introduce 3D content into for example the home environment. Extensions to this format have been proposed to enable occlusion handling, by introducing more image-and-depth layers, containing image data located behind the foreground objects, to enable looking around such foreground objects. In the presence of such a multi-layer representation, a next extension is to add transparency information to the layers, allowing for alpha matting (offering significant gains in rendering quality) and inclusion of semi-transparent objects in the scene. Due to the multi-layer nature, it is then still possible to tune the amount of depth. In this paper, we report on our design choices to arrive at the Declipse 2 format, by going through the whole video chain, from content creation, to looking at the effects of video compression and finally real-time 3D rendering and display of such multi-layer content with transparency information, showing the feasibility of the approach under practical conditions.