Local object-based super-resolution mosaicing from low-resolution video

Many efficient super-resolution methods have been presented in the past for improving resolution of images degraded by global blurs. Unfortunately, in video, more complex situations can be observed where local blurs appear in each frame which are mainly caused by object motions. To address this problem, we propose in this article a local super-resolution method which allows the restoration of such local blurs. Moreover, the motion of objects in video sequences may be very complex and particularly in very low-resolution sequences it is difficult to estimate their motion exactly enough to superimpose them for super-resolution. To this end, we present a generic method: An interpolation method is proposed to improve the resolution of moving objects and we derive from this a super-resolution method for the scene background.

[1]  Deepa Kundur,et al.  Blind image deconvolution revisited , 1996 .

[2]  Xu Liu,et al.  Document Image Mosaicing with Mobile Phones , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[3]  Nirmal K. Bose,et al.  Superresolution and noise filtering using moving least squares , 2006, IEEE Transactions on Image Processing.

[4]  P. H. Cittert.  Zum Einfluß der Spaltbreite auf die Intensitätsverteilung in Spektrallinien , 1930 .

[5]  I Ideses,et al.  Superresolution in turbulent videos: making profit from damage. , 2007, Optics letters.

[6]  Aggelos K. Katsaggelos,et al.  High-resolution images from low-resolution compressed video , 2003, IEEE Signal Process. Mag..

[7]  A. Murat Tekalp,et al.  Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time , 1997, IEEE Trans. Image Process..

[8]  Gerard de Haan,et al.  Adaptive Image Restoration Based on Local Robust Blur Estimation , 2007, ACIVS.

[9]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[10]  Joost van de Weijer,et al.  Fast Anisotropic Gauss Filtering , 2002, ECCV.

[11]  Frédéric Champagnat,et al.  Fast super-resolution on moving objects in video sequences , 2008, 2008 16th European Signal Processing Conference.

[12]  Yuan Jian.  A Fast Super-resolution Reconstruction Algorithm , 2006 .

[13]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[14]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[15]  Li Chen,et al.  Blind Super-Resolution Image Reconstruction using a Maximum a Posteriori Estimation , 2006, 2006 International Conference on Image Processing.

[16]  A. Murat Tekalp,et al.  Robust methods for high-quality stills from interlaced video in the presence of dominant motion , 1997, IEEE Trans. Circuits Syst. Video Technol..

[17]  A. Murat Tekalp,et al.  Robust, object-based high-resolution image reconstruction from low-resolution video , 1997, IEEE Trans. Image Process..

[18]  Aggelos K. Katsaggelos,et al.  Region-based super-resolution for compression , 2007, Multidimens. Syst. Signal Process..

[19]  Michael Elad,et al.  A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur , 2001, IEEE Trans. Image Process..

[20]  H. Joel Trussell,et al.  Identification and restoration of spatially variant motion blurs in sequential images , 1992, IEEE Trans. Image Process..

[21]  B. Gunturk,et al.  Multiframe resolution-enhancement methods for compressed video , 2002, IEEE Signal Processing Letters.

[22]  Jan Flusser,et al.  A Unified Approach to Superresolution and Multichannel Blind Deconvolution , 2007, IEEE Transactions on Image Processing.

[23]  Yasuyuki Matsushita,et al.  Removing Non-Uniform Motion Blur from Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[24]  Iain E. G. Richardson,et al.  H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia , 2003 .

[25]  Jiaya Jia,et al.  Single Image Motion Deblurring Using Transparency , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  A. Murat Tekalp,et al.  Bi-directional 2-D mesh representation for video object rendering, editing and superresolution in the presence of occlusion , 2003, Signal Process. Image Commun..

[27]  Russell C. Hardie,et al.  A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter , 2007, IEEE Transactions on Image Processing.

[28]  Klamer Schutte,et al.  Super-Resolution on Moving Objects and Background , 2006, 2006 International Conference on Image Processing.

[29]  Lisimachos P. Kondi,et al.  A regularization framework for joint blur estimation and super-resolution of video sequences , 2005, IEEE International Conference on Image Processing 2005.

[30]  Shree K. Nayar,et al.  Video super-resolution using controlled subpixel detector shifts , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Steven W. Smith,et al.  The Scientist and Engineer's Guide to Digital Signal Processing , 1997 .

[32]  P. H. Cittert.  Zum Einfluß der Spaltbreite auf die Intensitätsverteilung in Spektrallinien. II , 1930 .

[33]  Xuelong Li,et al.  A multi-frame image super-resolution method , 2010, Signal Process..

[34]  Guillermo Sapiro,et al.  A Variational Framework for Simultaneous Motion Estimation and Restoration of Motion-Blurred Video , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[35]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[36]  Boris Mansencal,et al.  Multiple Moving Object Detection for Fast Video Content Description in Compressed Domain , 2008, EURASIP J. Adv. Signal Process..

[37]  Yehoshua Y. Zeevi,et al.  Quasi Maximum Likelihood Blind Deconvolution of Images Using Optimal Sparse Representations , 2003 .

[38]  Steve McLaughlin,et al.  Comparative study of textural analysis techniques to characterise tissue from intravascular ultrasound , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[39]  Montse Pardàs,et al.  Deleted DOI: Audiovisual Head Orientation Estimation with Particle Filtering in Multisensor Scenarios , 2008 .

[40]  Ofer Hadar,et al.  Super-resolution mosaicing from MPEG compressed video , 2005, IEEE International Conference on Image Processing 2005.

[41]  Thomas Sikora,et al.  Super-resolution Mosaicing for Object based Video Format Conversion , 2006 .

[42]  Peyman Milanfar,et al.  A computationally efficient superresolution image reconstruction algorithm , 2001, IEEE Trans. Image Process..

[43]  Aggelos K. Katsaggelos,et al.  Tracking Motion-Blurred Targets in Video , 2006, 2006 International Conference on Image Processing.

[44]  Andrew Blake,et al.  Motion Deblurring and Super-resolution from an Image Sequence , 1996, ECCV.

[45]  Dianne P. O'Leary,et al.  Restoring Images Degraded by Spatially Variant Blur , 1998, SIAM J. Sci. Comput..

[46]  Takahiro Saito,et al.  Super-resolution sharpening-demosaicking method for removing image blurs caused by an optical low-pass filter , 2005, IEEE International Conference on Image Processing 2005.

[47]  Jenny Benois-Pineau,et al.  Interactive Fine Object-based Segmentation of Generic Video Scenes For Object-Based Indexing , 2003 .

[48]  Jan Flusser,et al.  Space-Variant Restoration of Images Degraded by Camera Motion Blur , 2008, IEEE Transactions on Image Processing.

[49]  Ofer Hadar,et al.  Super-resolution mosaicing from MPEG compressed video , 2005, ICIP.

[50]  Michael Elad,et al.  Super-Resolution Without Explicit Subpixel Motion Estimation , 2009, IEEE Transactions on Image Processing.

[51]  Klamer Schutte,et al.  Multiframe Super-Resolution Reconstruction of Small Moving Objects , 2010, IEEE Transactions on Image Processing.

[52]  Truong Q. Nguyen,et al.  LCD Motion Blur Reduction: A Signal Processing Approach , 2008, IEEE Transactions on Image Processing.

[53]  P. Jansson.  Method for Determining the Response Function of a High-Resolution Infrared Spectrometer* , 1970 .

[54]  Aggelos K. Katsaggelos,et al.  Iterative restoration of fast‐moving objects in dynamic image sequences , 1996 .