Real time turbulent video perfecting by image stabilization and super-resolution

Image and video quality in Long Range Observation Systems (LOROS) suffer from atmospheric turbulence that causes small neighbourhoods in image frames to chaotically move in different directions and substantially hampers visual analysis of such image and video sequences. The paper presents a real-time algorithm for perfecting turbulence degraded videos by means of stabilization and resolution enhancement. The latter is achieved by exploiting the turbulent motion. The algorithm involves generation of a "reference" frame and estimation, for each incoming video frame, of a local image displacement map with respect to the reference frame; segmentation of the displacement map into two classes: stationary and moving objects and resolution enhancement of stationary objects, while preserving real motion. Experiments with synthetic and real-life sequences have shown that the enhanced videos, generated in real time, exhibit substantially better resolution and complete stabilization for stationary objects while retaining real motion.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Rabab Kreidieh Ward,et al.  Restoration of randomly blurred images by the Wiener filter , 1989, IEEE Trans. Acoust. Speech Signal Process..

[4]  A. Baram,et al.  Restoration of turbulence-degraded images by the most-common method. , 1991, Applied optics.

[5]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[6]  A. Murat Tekalp,et al.  Efficient multiframe Wiener restoration of blurred and noisy image sequences , 1992, IEEE Trans. Image Process..

[7]  Its'hak Dinstein,et al.  Generation of a restored image from a video sequence recorded under turbulence effects , 1997 .

[8]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[9]  E. Bourennane,et al.  Restoration of images degraded by the atmospheric turbulence , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[10]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Leonid P. Yaroslavsky,et al.  Nonlinear filters for image processing in neuro-morphic parallel networks , 2003 .

[12]  Andrew Zisserman,et al.  Computer vision applied to super resolution , 2003, IEEE Signal Process. Mag..

[13]  Graham C. Goodwin,et al.  Super-resolution reconstruction using spatio-temporal filtering , 2003, J. Vis. Commun. Image Represent..

[14]  L Yaroslavsky,et al.  Boundary effect free and adaptive discrete signal sinc-interpolation algorithms for signal and image resampling. , 2003, Applied optics.

[15]  Barak Fishbain,et al.  Restoration of atmospheric turbulent video containing real motion using rank filtering and elastic image registration , 2004, 2004 12th European Signal Processing Conference.

[16]  Leonid Yaroslavsky.  Digital Holography and Digital Image Processing , 2004, Springer US.

[17]  Leonid P. Yaroslavsky,et al.  Nonlinear Filters in Image Processing , 2004 .

[18]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[19]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[20]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[21]  Nahum Kiryati,et al.  Piecewise-Smooth Dense Optical Flow via Level Sets , 2006, International Journal of Computer Vision.

[22]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[23]  Michael Elad,et al.  Multiframe demosaicing and super-resolution of color images , 2006, IEEE Transactions on Image Processing.

[24]  Barak Fishbain,et al.  Real time stabilization of long range observation system turbulent video , 2007, Electronic Imaging.