Low complexity hardware oriented H.264/AVC motion estimation algorithm and related low power and low cost architecture design

The ever increasing bit-rate on network applications such as broadcasting digital television makes storage capacity larger than ever before. Especially, the advent of Super Hi-Vision (SHV) which has feature of high resolution further intensifies the tough situation. Since limitation exists in network bandwidth and disk storage, the video compression technique is becoming more important than before. As the latest video coding standard, H.264/AVC can provide superior performance to previous standards. However, it also consists of huge complexity. When ASIC (Application Specific Integration Circuits) based real-time hardware system is considered, the intensive complexity in H.264/AVC will cause problems in hardware cost and power consumption. Therefore, to solve the problem, this dissertation focuses on two key issues which are low complexity hardware oriented algorithm and its related architecture. In H.264/AVC based system, motion estimation (ME) which is the major part of inter prediction is the most significant component. It consists of integer ME (IME) and fractional ME (FME) and occupies almost 90% computation, which makes it a must to divide IME and FME into two separate stages in real-time hardwired encoder. Besides motion estimation part, hardware engine of intra prediction is another time consuming part because of its abundant prediction modes. Moreover, the rate distortion based mode decision part which makes a final judgment of inter and intra modes also consumes lot of computation in the final stage of whole encoding system. Many software based fast algorithms have already been proposed to release complexity of H.264/AVC based system. However, most of these algorithms can not be efficiently realized in hardware because of constraints in hardware design. In hardware, factors such as predictable data flow, regular access of memory and full hardware utilization are important to the whole system’s performance. Without considering these factors, hardware cost, throughput and power consumption will increase greatly. So, hardware oriented low complexity algorithm and related low cost and low power hardware architecture are important issues to H.264/AVC based real-time encoder design. Based on analysis of existing works and current problem, this dissertation mainly targets on low cost and low power H.264/AVC real-time hardwired encoder. In detail, it focuses on IME, FME, intra and mode decision, which are four computation intensive parts in H.264/AVC based system. Firstly, low complexity algorithm which follows hardware data flow is proposed. Secondly, based on proposed algorithm, flexible and highly parallel architectures are given out. Moreover, architecture and circuit optimizations are proposed to further reduce the hardware cost and power consumption. The whole dissertation consists of 6 chapters as follows. In the first chapter, introduction in video compression field is given out. The development and feature of video coding standards and emphasis of this dissertation are described in detail. In the second chapter, hardware oriented low complexity motion estimation algorithms are given out. The complexity reduction is achieved in MRF, search range and matching pattern of H.264/AVC based system. Firstly, for MRF technique, gradient and block matching information are used for fast MRF algorithms. The proposed algorithms release the MRF complexity according to macroblock (MB) features in spatial and temporal domains. Secondly, based on the statistical analysis, it is shown that motion feature is conformity across several frames and search range can be adaptive adjusted according to the motion feature of MB. So, two proposals of search range adjustment is given out in this dissertation. For MB with extreme small motion, search range is restricted into 1/8 of original value. For MB with other cases, the search range is adjusted recursively according to the motion feature of MB on previous frame. Thirdly, since pixel difference can reflect spatial feature of current MB, it is used to classify matching pattern of ME process. An pixel difference based adaptive sub-sampling scheme is proposed, which uses three hardware oriented patterns for MB with different spatial features. By combining all the proposed schemes, the overall algorithm can achieve up to 95.72% complexity reduction with average 0.072dB PSNR loss and 0.902% bit-rate increase based on hardware data flow. In the third chapter, two flexible IME architectures for adaptive sub-sampling algorithm, namely adaptive propagate partial SAD (APPSAD) and reconfigurable SAD Tree (RSADT), are proposed. By using configurable SAD, the proposed RSADT architecture achieves data organizations in both architectural and memory level, which speeds up processing time and saving power consumption. For APPSAD, the original processing element (PE) is expanded into four different types. According to different matching patterns, only the related type of PE is enabled and power consumption of other types of PE can be saved. Moreover, circuit optimization is applied on both APPSAD and RSADT are optimized. The propagation chain, original PE and adder trees are simplified, with no redundant registers and adders. So, hardware cost and power consumption are further reduced. With TSMC 0.18um CMOS library, it is shown that the proposed architectures can achieve 61.71% saving of processing cycles and up to 39.8% power reduction of existing works. In the fourth chapter, two low design effort SHV engines for FME and intra prediction are proposed. Firstly, for FME engine, two optimizations in the algorithm level, namely inter mode pre-filtering and one-pass algorithm are proposed. For inter mode pre-filtering, it analyze the motion cost of sub-blocks in IME stage and only focuses on two modes which have smaller cost than others. As for one-pass algorithm, it firstly decides the sliding window based on integer motion cost of neighboring positions. Then, only half and quarter pixel within the sliding window are processed simultaneously, which saves hardware cost and processing time. In the hardware level, with quarter sub-sampling technique in FME stage, a 16-Pel interpolation structure is proposed, which speeds up 4 times of original 4-Pel design while keep almost the same hardware amount. With MB and frame level parallel processing flow, compared with representative design which requires 2.16GHz for 4k×4k@60fps, the proposed FME engine can accomplish real-time processing with only 145MHz. For intra engine, the predictor generation is the most time consuming part. From the analysis of data dependency issue of intra prediction, it is observed that the maximum parallel processing scale is two sub-block instead of original one sub-block way. In this dissertation, one lossless two sub-block parallel data flow are proposed, which saves 37.5% processing time of original one sub-block way. Also, in the original intra predictor generation engine, lots of repetitive computation exists among different modes. In the proposed fully utilized intra predictor generation architecture, no repetitive generation of predictors exists and it is applicable for all intra prediction modes. With proposed architecture, the whole predictor generation process can be finished within only 22.5% cycles of original design. By combining parallel data flow and fully utilized architecture, the proposed intra predictor generation engine is capable of handling 4k×2k@60fps specification. In the fifth chapter, high complexity problem in H.264/AVCmode decision is discussed. By utilizing spatial and temporal information, complexity reduction is achieved in two stages. Firstly, gradients of current MB and motion vector of encoded MB on both current and previous frames are utilized for pre-stage skip mode check. Secondly, during the motion stage, it is observed that information of motion vector predictor (MVP), block overlapping status and rate distortion cost can indicate the accuracy of matching process. In detail, the MVP represents the accuracy of predicted start point. The block overlapping status of different inter modes indicates the motion trend of object. As for rate distortion cost, it is an objective measurement of matching result. Thus, such information is used for early decision of whole encoding process in the proposed mode decision algorithm. Compared with existing works, the proposed algorithm can achieve up to 53.4% speed-up ratio with trivial quality loss. In the sixth chapter, the whole dissertation is concluded and future trend in video compression fields is also briefly discussed. In this dissertation, it focuses on IME, FME, intra and mode decision which are four most important parts in H.264/AVC real-time encoding system. Hardware oriented low complexity algorithm and low cost, low power hardware architectures are proposed. By combining hardware oriented algorithms with proposed architectures, compared with recent 4-stage real-time encoder design, about 90.68% power in IME part can be reduced. As for SHV targeted FME and intra engines, about 93.31% and 67.24% estimated power reduction in hardware design.