Sparse super-resolution reconstructions of video from mobile devices in digital TV broadcast applications

We consider the mobile service scenario where video programming is broadcast to low-resolution wireless terminals. In such a scenario, broadcasters utilize simultaneous data services and bi-directional communications capabilities of the terminals in order to offer substantially enriched viewing experiences to users by allowing user participation and user tuned content. While users immediately benefit from this service when using their phones in mobile environments, the service is less appealing in stationary environments where a regular television provides competing programming at much higher display resolutions. We propose a fast super-resolution technique that allows the mobile terminals to show a much enhanced version of the broadcast video on nearby high-resolution devices, extending the appeal and usefulness of the broadcast service. The proposed single frame super-resolution algorithm uses recent sparse recovery results to provide high quality and high-resolution video reconstructions based solely on individual decoded frames provided by the low-resolution broadcast.

[1]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[2]  Louis Dunn Fielder,et al.  ISO/IEC MPEG-2 Advanced Audio Coding , 1997 .

[3]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[4]  Sheila S. Hemami,et al.  Regularity-preserving image interpolation , 1999, IEEE Trans. Image Process..

[5]  Michael Elad,et al.  Super-Resolution Reconstruction of Image Sequences , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  M. Orchard,et al.  New edge-directed interpolation , 2001, IEEE Trans. Image Process..

[8]  Aggelos K. Katsaggelos,et al.  Bayesian high-resolution reconstruction of low-resolution compressed video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  A. Willsky Multiresolution Markov models for signal and image processing , 2002, Proc. IEEE.

[10]  Yücel Altunbasak,et al.  Super-resolution still and video reconstruction from MPEG-coded video , 2002, IEEE Trans. Circuits Syst. Video Technol..

[11]  Albert Cohen,et al.  Edge adapted nonlinear multiscale transforms for compact image representation , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[12]  Thomas Wiegand,et al.  Draft ITU-T recommendation and final draft international standard of joint video specification , 2003 .

[13]  Onur G. Guleryuz,et al.  Weighted overcomplete denoising , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[14]  Onur G. Guleryuz Predicting wavelet coefficients over edges using estimates based on nonlinear approximants , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[15]  Michael Elad,et al.  Fast and Robust Multi-Frame Super-Resolution , 2004, IEEE Transactions on Image Processing.

[16]  Onur G. Guleryuz,et al.  A nonlinear loop filter for quantization noise removal in hybrid video compression , 2005, IEEE International Conference on Image Processing 2005.

[17]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[18]  Peyman Milanfar,et al.  Statistical performance analysis of super-resolution , 2006, IEEE Transactions on Image Processing.

[19]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[20]  Martin Vetterli,et al.  Locally adaptive wavelet-based image interpolation , 2006, IEEE Transactions on Image Processing.

[21]  O. Yamamori,et al.  Mobile Terminal Supporting Terrestrial Digital TV Broadcasting , 2007 .