Rate-distortion Optimized Streaming of Compressed Light Fields with Multiple Representations

Light field rendering has been proposed as a way of enabling interactive photorealistic viewing of objects and scenes without the complexity of traditional computer graphics rendering techniques. Light field rendering, however, relies on a large amount of image data to achieve photorealistic quality and freedom in viewing directions and positions. In order to reduce the size of the data set, efficient compression is used. We focus on one class of compression algorithms that uses closed-loop prediction of the light field images. For remote viewing over a network, compressed light field data sets can be streamed to an interacting user. A rate-distortion optimized packet scheduling framework has been proposed for interactive streaming of compressed light fields. Experiments show that for several streaming scenarios and data sets, better streaming performance can be obtained by using independent encoding of the images instead of prediction. One reason for this is that using prediction restricts random access and can possibly increase the rate for transmitting the required set of images. Recently, we have proposed a light field coding scheme that uses multiple representations to provide similar random access capabilities as independent encoding, but with better compression efficiency. In this paper, we extend the rate-distortion optimized streaming framework designed for conventionally encoded light fields to this new multiple representations encoding scheme. Experimental results show that the rate-distortion streaming performance with multiple representations is superior to that using independent encoding of images.