FPGA implementation of a real-time super-resolution system using a convolutional neural network

Super-resolution technologies are used to fill the gap between high-resolution displays and lower-resolution contents. There are various algorithms to interpolate information, one of which is using a convolutional neural network (CNN). This paper shows FPGA implementation and performance evaluation of a CNN-based super-resolution system, which can process moving images in real time. We apply horizontal and/or vertical flips to network input images instead of commonly used pre-enlargement techniques. This method prevents information loss and enables the network to utilize the best of its input image size. Our system can perform super-resolution from 960×540 pixels to 1920×1080 pixels at not less than 48fps with a latency of less than 1 ms. Even though the network scale and the size of filters are limited due to resource restriction of the FPGA, the system generates clear super-resolution images with smooth edges. The evaluation results also reveal that the proposed system achieves superior quality in terms of the structural similarity (SSIM) index, compared to other systems using pre-enlargement.