Hardware implementation of real-time multiple frame super-resolution

Super-resolution reconstruction is a method for reconstructing higher resolution images from a set of low resolution observations. The sub-pixel differences among different observations of the same scene allow to create higher resolution images with better quality. In the last thirty years, many methods for creating high resolution images have been proposed. However, hardware implementations of such methods are limited. In this work, highly parallel and pipelined implementation for iterative back projection super-resolution algorithm is presented. The proposed hardware implementation is capable of reconstructing 512×512 sized images from set of 20 lower resolution observations, with real-time capabilities up to 25 frame per second (fps). Explained system has been synthesized and verified via Xilinx VC707 FPGAs. To the best of our knowledge, the system is currently the fastest super-resolution implementation based on FPGA.

[1]  Gustavo Marrero Callicó,et al.  Efficient FPGA implementation of a high-quality super-resolution algorithm with real-time performance , 2011, IEEE Transactions on Consumer Electronics.

[2]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[3]  Michael Unser,et al.  Convolution-based interpolation for fast, high-quality rotation of images , 1995, IEEE Trans. Image Process..

[4]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[5]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[6]  Christos-Savvas Bouganis,et al.  Real-time image super resolution using an FPGA , 2008, 2008 International Conference on Field Programmable Logic and Applications.

[7]  Peyman Milanfar,et al.  Super-resolution imaging , 2011 .

[8]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Maria E. Angelopoulou,et al.  FPGA-based Real-time Super-Resolution on an Adaptive Image Sensor , 2008, ARC.

[10]  Vivek Bannore.  Iterative-Interpolation Super-Resolution (IISR) , 2009 .

[11]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[12]  Sebastián López,et al.  Analysis of fast block matching motion estimation algorithms for video super-resolution systems , 2008, IEEE Transactions on Consumer Electronics.

[13]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[15]  Leon Hirsch,et al.  Super Resolution From A Single Image , 2016 .

[16]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[17]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[18]  Kiyoharu Aizawa,et al.  Signal-processing based method for acquiring very high resolution images with multiple cameras and its theoretical analysis , 1992 .

[19]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[20]  Shmuel Peleg,et al.  Image sequence enhancement using sub-pixel displacements , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[22]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[23]  Albert J. P. Theuwissen,et al.  CMOS image sensors: State-of-the-art , 2008 .

[24]  Luis Alvarez,et al.  Motion Estimation Techniques in Super-Resolution Image Reconstruction. A Performance Evaluation , 2006 .