Error Concealment in Encoded Video Streams

In ATM networks cell loss causes data to be dropped, which results in the loss of entire macroblocks when MPEG video is being transmitted. In order to reconstruct the missing data, the location of these macroblocks must be known. We describe a technique for packing ATM cells with compressed data, with the aim of detecting the location of missing macroblocks in the encoded video stream. This technique also permits proper decoding of correctly received macroblocks, and thus prevents the loss of ATM cells from affecting the decoding process. We also describe spatial and temporal techniques for the recovery of lost macroblocks. The spatial techniques fall into two categories: deterministic and statistical. A deterministic spatial approach we provide aims at reconstructing each lost pixel by spatial interpolation from the nearest undamaged pixels. Another, recovers lost macroblocks by minimizing inter-sample variations within each block and across its boundaries. In the statistical approach, each frame is modeled as a Markov Random Field, and a maximum a posteriori (MAP) estimate of the missing macroblocks is obtained based on this model. The MAP estimate for each pixel within a lost macroblock is obtained by means of the iterative conditional modes (ICM) algorithm. The iterative method is guaranteed to converge to a global maximum, even though the global maximum is not unique. It is shown that, for each pixel, the median of its neighbors is a MAP estimate. In temporal reconstruction, a search is carried out over a reference frame for the macroblock sized region that will maximize the posterior distribution of the lost macroblock given its neighbors.