Superresolution imaging: a survey of current techniques

Imaging plays a key role in many diverse areas of application, such as astronomy, remote sensing, microscopy, and tomography. Owing to imperfections of measuring devices (e.g., optical degradations, limited size of sensors) and instability of the observed scene (e.g., object motion, media turbulence), acquired images can be indistinct, noisy, and may exhibit insuffcient spatial and temporal resolution. In particular, several external effects blur images. Techniques for recovering the original image include blind deconvolution (to remove blur) and superresolution (SR). The stability of these methods depends on having more than one image of the same frame. Differences between images are necessary to provide new information, but they can be almost unperceivable. State-of-the-art SR techniques achieve remarkable results in resolution enhancement by estimating the subpixel shifts between images, but they lack any apparatus for calculating the blurs. In this paper, after introducing a review of current SR techniques we describe two recently developed SR methods by the authors. First, we introduce a variational method that minimizes a regularized energy function with respect to the high resolution image and blurs. In this way we establish a unifying way to simultaneously estimate the blurs and the high resolution image. By estimating blurs we automatically estimate shifts with subpixel accuracy, which is inherent for good SR performance. Second, an innovative learning-based algorithm using a neural architecture for SR is described. Comparative experiments on real data illustrate the robustness and utilization of both methods.

[1]  Yoram Bresler,et al.  Perfect blind restoration of images blurred by multiple filters: theory and efficient algorithms , 1999, IEEE Trans. Image Process..

[2]  Shree K. Nayar,et al.  Motion-based motion deblurring , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jan Flusser,et al.  Multiframe blind deconvolution coupled with frame registration and resolution enhancement , 2007 .

[4]  Subhasis Chaudhuri,et al.  An MRF Model-Based Approach to Simultaneous Recovery of Depth and Restoration from Defocused Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  S. Gabarda,et al.  Blind image quality assessment through anisotropy. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[6]  Jan Flusser,et al.  Multichannel blind iterative image restoration , 2003, IEEE Trans. Image Process..

[7]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[8]  Hung-Ta Pai,et al.  On eigenstructure-based direct multichannel blind image restoration , 2001, IEEE Trans. Image Process..

[9]  Gjlles Aubert,et al.  Mathematical problems in image processing , 2001 .

[10]  Nikolas P. Galatsanos,et al.  EM-based simultaneous registration, restoration, and interpolation of super-resolved images , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[12]  Mohammad S. Alam,et al.  Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames , 2000, IEEE Trans. Instrum. Meas..

[13]  J. Nagy,et al.  Numerical methods for coupled super-resolution , 2006 .

[14]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[15]  Aggelos K. Katsaggelos,et al.  Bayesian resolution enhancement of compressed video , 2004, IEEE Transactions on Image Processing.

[16]  Patrizio Campisi,et al.  Multichannel blind image deconvolution using the Bussgang algorithm: spatial and multiresolution approaches , 2003, IEEE Trans. Image Process..

[17]  Andrew E. Yagle Blind superresolution from undersampled blurred measurements , 2003, SPIE Optics + Photonics.

[18]  Henri Maitre,et al.  Multi-channel high resolution blind image restoration , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[19]  Yuanxu Chen,et al.  A general approach to blind image super-resolution using a PDE framework , 2005, Visual Communications and Image Processing.

[20]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[21]  Jeremy R. Cooperstock,et al.  Fast Super-Resolution for Rational Magnification Factors , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[23]  Deepu Rajan,et al.  Simultaneous Estimation of Super-Resolved Scene and Depth Map from Low Resolution Defocused Observations , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[25]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[26]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[27]  Francisco de Borja Rodríguez Ortiz,et al.  A two-step neural-network based algorithm for fast image super-resolution , 2007, Image Vis. Comput..

[28]  Nikolas P. Galatsanos,et al.  Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images , 2006, IEEE Transactions on Image Processing.

[29]  Yaron Caspi,et al.  Under the supervision of , 2003 .

[30]  Jan Flusser,et al.  Resolution enhancement via probabilistic deconvolution of multiple degraded images , 2006, Pattern Recognit. Lett..

[31]  Jan Flusser,et al.  A Unified Approach to Superresolution and Multichannel Blind Deconvolution , 2007, IEEE Transactions on Image Processing.

[32]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[33]  Timothy J. Holmes,et al.  Subpixel deconvolution of 3D optical microscope imagery , 2004, SPIE Optics + Photonics.

[34]  Ali Mohammad-Djafari,et al.  Super-Resolution Using Hidden Markov Model and Bayesian Detection Estimation Framework , 2006, EURASIP J. Adv. Signal Process..

[35]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[36]  Jan Flusser,et al.  Super-Resolution and Blind Deconvolution For Rational Factors With an Application to Color Images , 2009, Comput. J..

[37]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[38]  Peyman Milanfar,et al.  Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement , 2001, IEEE Trans. Image Process..

[39]  Mostafa Kaveh,et al.  Blind image restoration by anisotropic regularization , 1999, IEEE Trans. Image Process..

[40]  D. Tschumperlé PDE's based regularization of multivalued images and applications , 2002 .

[41]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[42]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[43]  Jan Flusser,et al.  Multichannel blind deconvolution of spatially misaligned images , 2005, IEEE Transactions on Image Processing.

[44]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[45]  Aggelos K. Katsaggelos,et al.  Parameter estimation in Bayesian high-resolution image reconstruction with multisensors , 2003, IEEE Trans. Image Process..

[46]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[47]  Terrance E. Boult,et al.  Efficient super-resolution via image warping , 2000, Image Vis. Comput..

[48]  Gerald C. Holst,et al.  Electro-Optical Imaging System Performance , 1995 .

[49]  Michael Elad,et al.  A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur , 2001, IEEE Trans. Image Process..

[50]  Russell C. Hardie,et al.  Aliasing reduction in staring infrared imagers utilizing subpixel techniques , 1995 .

[51]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .