Content-based selective enhancement for streaming video

Video transmission over bandwidth-varying networks is becoming increasingly important due to emerging applications such as streaming of video over the Internet. The fundamental obstacle in designing such systems resides in the varying characteristics of the Internet (i.e. bandwidth variations and packet-loss patterns). In MPEG-4, a new scalability scheme, called fine-granular-scalability (FGS), was standardized, which is able to adapt to bandwidth variations in real-time while using the same pre-encoded stream. This paper presents a novel technique that enables the FGS coding scheme to perform content-based enhancement and prioritized transmission of specific regions. The proposed selective enhancement mechanism gives the FGS framework the flexibility to perform differentiated bit-allocation to enhance specific objects, such that an improved visual image quality can be obtained at various bit-rates. In our system, we employed a novel real-time face detection algorithm, that in conjunction with the proposed content-based selective enhancement method consistently leads to better subjective visual quality of streaming video under various transmission bit-rates. The FGS selective enhancement method presented has been adopted in the MPEG-4 standard.