Algorithm and Architecture Design of Multirate Frame Rate Up-conversion for Ultra-HD LCD Systems

In current liquid crystal display (LCD) systems, the frame size becomes larger than the ultra-HD ( $3840\times 2160$ ) resolution, and the refresh rate becomes higher than 120 Hz or more. However, the available video frame rates are usually at 24, 30, or 60 frames/s only, which are lower than the refresh rate of LCDs. To fill the gap between the video and LCD systems, frame interpolation techniques are usually adopted. Although frame rate up-conversion (FRUC) is regarded as the most efficient method, many design challenges are encountered in the current high-resolution and high-frame-rate LCD systems. In this paper, we developed a hardware-efficient multirate FRUC, which is capable of increasing the video frame rate from 24 or 60 to 120 frames/s. We improved the accuracy of motion vectors (MVs) between the video frames using predictive square search motion estimation (ME), followed by Markov random field (MRF) correction. Subsequently, we applied the block-based forward motion compensation (MC) to interpolate the intermediate frames. Thereafter, we performed subblock refinement to enhance the visual quality. The experiments show that the proposed algorithm performed well in both subjective and objective evaluations. We also designed hardware architecture for our multirate FRUC to support the current ultra-HD LCD systems. We proposed ping-pong two-way scheduling to eliminate the dependence among blocks. We realized 54%, 70%, and 35% cycle reduction and 62%, 82%, and 35% bandwidth reduction in the ME, MRF MV correction, and MC, respectively. The SRAM was shared by all modules, and its total size was reduced by 88%. We also implemented our design to a chip using the TSMC 90-nm cell library.