Enhancing Multi-Frame Images Using Self-Delaying Dynamic Networks

This paper presents the use of a newly created network structure known as a Self-Delaying Dynamic Network (SDN) to create a high resolution image from a set of time stepped input frames. These SDNs are non-recurrent temporal neural networks which can process time sampled data. SDNs can store input data for a lifecycle and feature dynamic logic based connections between layers. Several low resolution images and one high resolution image of a scene were presented to the SDN during training by a Genetic Algorithm. The SDN was trained to process the input frames in order to recreate the high resolution image. The trained SDN was then used to enhance a number of unseen noisy image sets. The quality of high resolution images produced by the SDN is compared to that of high resolution images generated using Bi-Cubic interpolation. The SDN produced images are superior in several ways to the images produced using Bi-Cubic interpolation. Keywords—Image Enhancement, Neural Networks, Multi-Frame.

[1]  Thomas Martin Deserno,et al.  Survey: interpolation methods in medical image processing , 1999, IEEE Transactions on Medical Imaging.

[2]  Yun Zhang,et al.  Multi-resolution and multi-spectral image fusion for urban object extraction , 2004 .

[3]  B. S. Manjunath,et al.  Multi-sensor image fusion using the wavelet transform , 1994, Proceedings of 1st International Conference on Image Processing.

[4]  Allen M. Waxman,et al.  Color Night Vision: Opponent Processing in the Fusion of Visible and IR Imagery , 1997, Neural Networks.

[5]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[6]  Michael Unser,et al.  Enlargement or reduction of digital images with minimum loss of information , 1995, IEEE Trans. Image Process..

[7]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[8]  A. Garzelli.  WAVELET-BASED FUSION OF OPTICAL AND SAR IMAGE DATA OVER URBAN AREA , 2002 .

[9]  E. Meijering.  A chronology of interpolation: from ancient astronomy to modern signal and image processing , 2002, Proc. IEEE.

[10]  E. Salari,et al.  Integrated recurrent neural network for image resolution enhancement from multiple image frames , 2003 .

[11]  DeLiang Wang,et al.  On Temporal Generalization of Simple Recurrent Networks , 1996, Neural Networks.

[12]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[13]  Max A. Viergever,et al.  Quantitative evaluation of convolution-based methods for medical image interpolation , 2001, Medical Image Anal..

[14]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[15]  J. Schott,et al.  Resolution enhancement of multispectral image data to improve classification accuracy , 1993 .

[16]  Alain Fournier,et al.  2D Image Reconstruction Using Natural Neighbour Interpolation , 2000, WSCG.

[17]  Gonzalo Pajares Martinsanz,et al.  A wavelet-based image fusion tutorial , 2004 .

[18]  E. Meijering,et al.  A chronology of interpolation: from ancient astronomy to modern signal and image processing , 2002, Proc. IEEE.

[19]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[20]  Neil A. Dodgson,et al.  Quadratic interpolation for image resampling , 1997, IEEE Trans. Image Process..

[21]  Wl Lau,et al.  Comparison of image data fusion techniques using entropy and INI , 2016 .

[22]  R. Ivry,et al.  The neural representation of time , 2004, Current Opinion in Neurobiology.

[23]  Jos Nijhuis,et al.  Neural network approaches to capture temporal information , 2001 .

[24]  Andreas V. M. Herz,et al.  How Is Time Represented in the Brain , 2006 .

[25]  Sebastiano Battiato,et al.  Analysis and characterization of super-resolution reconstruction methods , 2003, IS&T/SPIE Electronic Imaging.

[26]  Erik H. W. Meijering,et al.  Spline interpolation in medical imaging: Comparison with other convolution-based approaches , 2000, 2000 10th European Signal Processing Conference.

[27]  Peter Caccetta,et al.  Image Fusion with Conditional Probability Networks for Monitoring the Salinization of Farmland , 1998, Digit. Signal Process..

[28]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[29]  Joonki Paik,et al.  An edge-preserving image interpolation system for a digital camcorder , 1996 .

[30]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[31]  J. Flusser,et al.  Image Fusion via Probabilistic Deconvolution , 2004 .

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.