High-Resolution Video Synthesis from Mixed-Resolution Video Based on the Estimate-and-Correct Method

A technique for increasing the frame rate of CMOS video cameras is presented. The method uses the non-destructive readout capabilities of CMOS imagers to obtain low-speed, high-resolution frames and high-speed, low-resolution frames simultaneously. The algorithm translates the pixels of the full resolution images with respect to the motion dynamics observed in the low-resolution frames and corrects the result as necessary for consistency with the low-resolution frames. Noting that due to the longer exposure time required, high-resolution frames are more prone to motion blur than low-resolution frames, and thus, a motion blur reduction step is also applied. Simulations demonstrate the ability of our technique in synthesizing high-quality, high-resolution frames at modest computational expense.

[1]  Larry H. Matthies,et al.  Multiresolution image sensor , 1997, IEEE Trans. Circuits Syst. Video Technol..

[2]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[3]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[4]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[5]  B. Pain,et al.  A CMOS imager with on-chip variable resolution for light-adaptive imaging , 1998, 1998 IEEE International Solid-State Circuits Conference. Digest of Technical Papers, ISSCC. First Edition (Cat. No.98CH36156).

[6]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[7]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[8]  Yaron Caspi,et al.  Increasing video resolution in tim and space , 2002, eccv 2002.

[9]  Michael Stegherr,et al.  Parameterizable VLSI architectures for the full-search block-matching algorithm , 1989 .

[10]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[11]  Peter Pirsch,et al.  VLSI architectures for hierarchical block matching algorithms , 1990, IEEE International Symposium on Circuits and Systems.

[12]  Gerald C. Holst,et al.  CCD arrays, cameras, and displays , 1996 .

[13]  Yi-Ping Hung,et al.  Fast block matching algorithm based on the winner-update strategy , 2001, IEEE Trans. Image Process..

[14]  Ming-Ting Sun,et al.  A family of vlsi designs for the motion compensation block-matching algorithm , 1989 .

[15]  Xinqiao Liu,et al.  Simultaneous image formation and motion blur restoration via multiple capture , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[16]  Aggelos K. Katsaggelos,et al.  Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images , 1995, Proceedings., International Conference on Image Processing.

[17]  Thomas S. Huang,et al.  Image processing , 1971 .

[18]  Peter Pirsch,et al.  Array architectures for block matching algorithms , 1989 .

[19]  Michael Elad,et al.  Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images , 1997, IEEE Trans. Image Process..