Rate Control and Error Resilience for Object-Based Video Coding

The MPEG-4 audiovisual coding standard introduced the object-based video data representation model where video data is no longer seen as a sequence of frames or fields, but consists of independent (semantically) relevant video objects that together build the video scene. This representation approach allows new and improved functionalities, but it has also created new relevant problems in terms of typical non-normative parts of the standard, such as rate control and error resilience, which need to be solved in order to successfully transmit object-based video with an acceptable quality over networks that have critical bandwidth and channel error characteristics, such as mobile networks and the Internet. To deal with the specific problems of object-based video coding, rate control demands two levels of action: 1) the scene-level, which is responsible for dynamically allocating the available resources between the various objects in the scene (i.e., between the different encoding time instants and the different video objects to encode in each time instant), aiming at minimizing quality variations along time and between the various objects in the scene; and 2) the object-level, which is responsible for allocating the resources attributed to each object among the various types of data to code (for that object), notably texture and shape, and for computing the best encoding parameters to achieve the target bit allocations while maintaining smooth quality fluctuations. In terms of error resilience techniques, the object-based coding approach means that shape and composition information also have to be taken into account for error resilience purposes, in addition to motion and texture data. To do this, at the encoder side, the coding of video objects is typically supervised by a resilience configuration module, which is responsible for choosing the most adequate coding parameters in terms of resilience for each video object. This is important because the decoding performance will much depend on the protective actions the encoder has taken. At the decoder side, defensive actions have to be taken. This includes error detection and error localization for each decoded video object, followed by independent object-level error concealment. Afterwards, advanced scene-level error concealment is also performed, which has access to all the video objects in the scene and is used immediately before the final concealed video scene is presented to the user. In this chapter, the most recent basics, advances and trends in terms of rate control and error resilience for object-based video coding will be described.