PIRM2018 Challenge on Spectral Image Super-Resolution: Dataset and Study

This paper introduces a newly collected and novel dataset (StereoMSI) for example-based single and colour-guided spectral image super-resolution. The dataset was first released and promoted during the PIRM2018 spectral image super-resolution challenge. To the best of our knowledge, the dataset is the first of its kind, comprising 350 registered colour-spectral image pairs. The dataset has been used for the two tracks of the challenge and, for each of these, we have provided a split into training, validation and testing. This arrangement is a result of the challenge structure and phases, with the first track focusing on example-based spectral image super-resolution and the second one aiming at exploiting the registered stereo colour imagery to improve the resolution of the spectral images. Each of the tracks and splits has been selected to be consistent across a number of image quality metrics. The dataset is quite general in nature and can be used for a wide variety of applications in addition to the development of spectral image super-resolution methods.

[1]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[2]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[3]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Sabine Süsstrunk,et al.  Multi-modal Spectral Image Super-Resolution , 2018, ECCV Workshops.

[5]  Cong Phuoc Huynh,et al.  Imaging Spectroscopy for Scene Analysis , 2012, Advances in Computer Vision and Pattern Recognition.

[6]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[7]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Peng Jiang,et al.  Hyperspectral face recognition based on spatio-spectral fusion and local binary pattern , 2017, Applied Optics and Photonics China.

[9]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[10]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[11]  Martial Hebert,et al.  Deep Material-Aware Cross-Spectral Stereo Matching , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[14]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Haida Liang,et al.  Advances in multispectral and hyperspectral imaging for archaeology and art conservation , 2012 .

[16]  Mac McKee,et al.  Estimating chlorophyll with thermal and broadband multispectral high resolution imagery from an unmanned aerial system using relevance vector machines for precision agriculture , 2015, Int. J. Appl. Earth Obs. Geoinformation.

[17]  Da-Wen Sun,et al.  Application of Hyperspectral Imaging in Food Safety Inspection and Control: A Review , 2012, Critical reviews in food science and nutrition.

[18]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  Di Wu,et al.  Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review — Part II: Applications , 2013 .

[20]  Alexander F. H. Goetz,et al.  Three decades of hyperspectral remote sensing of the Earth: a personal view. , 2009 .

[21]  Radu Timofte,et al.  NTIRE 2018 Challenge on Spectral Reconstruction from RGB Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Joaquim Salvi,et al.  Review of CMOS image sensors , 2006, Microelectron. J..

[23]  Yunsong Li,et al.  Hyperspectral image super-resolution using deep convolutional neural network , 2017, Neurocomputing.

[24]  Luc Van Gool,et al.  NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Radu Timofte,et al.  PIRM2018 Challenge on Spectral Image Super-Resolution: Methods and Results , 2018, ECCV Workshops.

[26]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[28]  Dong Liu,et al.  Deep Residual Attention Network for Spectral Image Super-Resolution , 2018, ECCV Workshops.

[29]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[30]  Sabine Süsstrunk,et al.  Multi-spectral SIFT for scene category recognition , 2011, CVPR 2011.

[31]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Mark R. Pickering,et al.  Multi-spectral remote sensing image registration via spatial relationship analysis on sift keypoints , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[33]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Abigail A. Fraeman,et al.  Multispectral Imaging of Mars from the Mars Science Laboratory Mastcam Instruments: Spectral Properties and Mineralogic Implications Along the Gale Crater Traverse , 2016 .

[35]  Kinjiro Amano,et al.  Information limits on neural identification of colored surfaces in natural scenes , 2004, Visual Neuroscience.

[36]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[37]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.