LoGSRN: Deep Super Resolution Network for Digital Elevation Model*

The spatial resolution of a Digital Elevation Model (DEM) plays a crucial role in many practical remote sensing applications. However, it is normally limited by the spatial resolution of the raw input imagery, from which a DEM is derived. One solution to enhance the limited resolution of a DEM during the post-processing, is fusing previously obtained high resolution DEM data. This data-driven approach appears particularly promising, considering the recent success of a deep convolutional network in single image super resolution (SISR). In this paper, we propose a new SISR network that can recover a high resolution DEM. Instead of configuring a single network directly mapping low resolution depth values to high resolution depth values, we propose a new model consisting of 3 subnetworks, i.e. a) extracting feature maps; b) inferring the high frequency details; c) refining the result combining the low resolution input with the details from b). This is similar to LapSRN [1] in that both adopt a Laplacian image pyramid to model the scaling process in SISR. However, the proposed method implements a much deeper subnetworks efficiently with multiple recursive feedback and feedforward connections, and an additional Laplacian of Gaussian (LoG)based loss function help to produce more effective training results. In this research, we also produce a high quality DEM dataset obtained from optical and lidar sensors, from satellites and aircraft respectively, covering different scenes found in remote sensing applications. Our experiments demonstrate that the proposed model performs better than other standard deep SISR models in terms of the training convergence and the Peak Signal to Noise Ratio (PSNR) of a reconstructed DEM.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Aggelos K. Katsaggelos,et al.  Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images , 1995, Proceedings., International Conference on Image Processing.

[3]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  S. Chaudhuri Super-Resolution Imaging , 2001 .

[5]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhenzhong Chen,et al.  Remote sensing image super-resolution: Challenges and approaches , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[10]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[12]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[13]  Susan Cox,et al.  Super-resolution imaging in live cells , 2015, Developmental biology.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  Narendra Ahuja,et al.  Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[17]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[18]  Shuiping Gou,et al.  Remote Sensing Image Super-Resolution Reconstruction Based on Nonlocal Pairwise Dictionaries and Double Regularization , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Biao Hou,et al.  Adaptive Super-Resolution for Remote Sensing Images Based on Sparse Representation With Global Joint Dictionary Model , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Yu Zhao,et al.  Fast and Accurate Image Super-Resolution Using a Combined Loss , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Andrew Zisserman,et al.  Computer vision applied to super resolution , 2003, IEEE Signal Process. Mag..

[22]  Feng Li,et al.  Super Resolution for Remote Sensing Images Based on a Universal Hidden Markov Tree Model , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Zhenzhong Chen,et al.  Remote Sensing Image Super-resolution: Challenges and Approaches (Invited Paper) , 2015 .

[24]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[27]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[28]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[29]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.