Fast mode decision for H.264/AVC based on local spatio-temporal coherency

In this paper, we propose a fast mode decision algorithm for H.264/AVC. It is based on the spatio-temporal coherency of the local neighborhood including and around the current block. We first build the histograms of the current block and the co-located block in the reference frame, respectively. If the difference between the two histograms is small, we use large-block-size modes. Otherwise, we subdivide the current block into four equal-sized sub-blocks and estimate the motion vector (MV) for each. In general, if there is a high degree of coherency between those MVs, we use large-block-size modes and otherwise small-block-size modes. In addition, we use the number of neighboring large blocks and the sub-blocks' ratedistortion (R-D) costs as further hints for the mode decision. As experimentally demonstrated, our algorithm leads to significant saving in computing time on the test video sequences.