Image Super-Resolution Reconstruction Based on Disparity Map and CNN

This paper presents a new method for registration of digital micro-mirror device (DMD) cameras through image reconstruction. The method called dual-channel super-resolution reconstruction (DCSR) uses a dual-channel input convolutional neural network method to iteratively obtain high resolution images. The originality of this study relies on the input color weight disparity map that enables extracting edges cues and feature cues effectively during the super-resolution. The iterative initial disparity map is obtained by a cost aggregation method based on the color weights, and the model parameters are updated during the iterative process. To validate performance, a series of the comparison experiments is developed on different network structures and parameter settings for the diverse deconv-pooling layers, convolution kernel sizes, and activation function. Experimental results show that the DCSR method can solve the sub-pixel registration problem of DMD cameras and achieve a good image reconstruction effect. The DCSR performs higher accuracy and an efficient process compared with traditional reconstruction made by single-channel input convolution neural network. The spatial resolution of the reconstructed image is significantly improved, and the detail resolution is enhanced.

[1]  Xing Mei,et al.  On building an accurate stereo matching system on graphics hardware , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[2]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[6]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Elias Kristensson,et al.  High dynamic spectroscopy using a digital micromirror device and periodic shadowing. , 2017, Optics express.

[8]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[9]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[10]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jianjun Lei,et al.  Color-Guided Depth Map Super Resolution Using Convolutional Neural Network , 2017, IEEE Access.

[12]  Motoharu Fujigaki,et al.  Accurate pixel-to-pixel correspondence adjustment in a digital micromirror device camera by using the phase-shifting moiré method. , 2006, Applied optics.

[13]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Adrian Travis,et al.  Dmd-based autostereoscopic display system for 3d interaction , 2008 .

[15]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Pascal Monasse,et al.  A Precision Analysis of Camera Distortion Models , 2017, IEEE Trans. Image Process..

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Andrew G. Howard,et al.  Some Improvements on Deep Convolutional Neural Network Based Image Classification , 2013, ICLR.

[19]  Yanbing Xue,et al.  Iterative color-depth MST cost aggregation for stereo matching , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[20]  Motoharu Fujigaki,et al.  Development of DMD reflection-type CCD camera for phase analysis and shape measurement , 2005, International Symposium on Optomechatronic Technologies.

[21]  Huili Grace Xing,et al.  Approaching real-time terahertz imaging with photo-induced coded apertures and compressed sensing , 2014 .

[22]  Kyungjae Lee,et al.  Brightness-Based Convolutional Neural Network for Thermal Image Enhancement , 2017, IEEE Access.

[23]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[24]  Fumin Zhang,et al.  Digital micromirror device camera with per-pixel coded exposure for high dynamic range imaging. , 2017, Applied optics.

[25]  Jongchul Lee,et al.  Improved census transform for noise robust stereo matching , 2016 .

[26]  Jie Li,et al.  Research on image reconstruction based and pixel unmixing based sub-pixel mapping methods , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[27]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.