Automatic Stitching for Hyperspectral Images Using Robust Feature Matching and Elastic Warp

Hyperspectral images, which contain not only spatial information but also rich spectral information, have been extensively applied to the fields of agriculture, urban planning, etc. However, it is difficult for a single image to cover a large area. Therefore, it requires to take photos of various parts and apply image stitching technology to obtain a panoramic hyperspectral image. When the viewpoint of the scene changes a lot, the ghost issue will occur with traditional methods. In order to get the high-precision resultant panoramas, this article proposes an automatic image stitching algorithm for hyperspectral images using robust feature matching and elastic warp. Our method contains two stages. The first stage is to choose one band as reference band and obtain the panorama in a single band. In particular, we extract feature points by scale-invariant feature transform. Then, we propose an efficient algorithm called multiscale top K rank preservation algorithm, for establishing robust point correspondences between two sets of points. Next, we adopt robust elastic warp to obtain the panorama of each band. The second stage is to stitch all remaining bands based on the transformation obtained in the first stage and fuse the information of all bands together to get the final panoramic hyperspectral image. Extensive experiments have demonstrated the effectiveness of our proposed method.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[3]  Jing Li,et al.  Parallax-Tolerant Image Stitching Based on Robust Elastic Warping , 2018, IEEE Transactions on Multimedia.

[4]  Sharath Pankanti,et al.  Adaptive as-natural-as-possible image stitching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yoichi Sato,et al.  Shape-Preserving Half-Projective Warps for Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[7]  Suhad Lateef Al-khafaji,et al.  Spectral-Spatial Scale Invariant Feature Transform for Hyperspectral Images , 2018, IEEE Transactions on Image Processing.

[8]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[9]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Fei Lei,et al.  A fast method for image mosaic based on SURF , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[11]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[12]  Yi Wan,et al.  Image Stitching with Weighted Elastic Registration , 2018, 2018 10th International Conference on Communications, Circuits and Systems (ICCCAS).

[13]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[14]  Md. Jan Nordin,et al.  Contour-Based Corner Detection and Classification by Using Mean Projection Transform , 2014, Sensors.

[15]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Dong Zhao,et al.  A Novel Mosaic Method for UAV-Based Hyperspectral Images , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[17]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[18]  Quan Z. Sheng,et al.  Nonrigid Point Set Registration With Robust Transformation Learning Under Manifold Regularization , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[19]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[20]  Junjun Jiang,et al.  Guided Locality Preserving Feature Matching for Remote Sensing Image Registration , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Md. Jan Nordin,et al.  Deformation invariant image matching based on dissimilarity of spatial features , 2016, Neurocomputing.

[22]  Mei Zhou,et al.  A Multi-Dimensional Hyperspectral Image Mosaic Method and its Acquisition System , 2018, 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[23]  Junjun Jiang,et al.  Robust Feature Matching Using Spatial Clustering With Heavy Outliers , 2020, IEEE Transactions on Image Processing.

[24]  Junjun Jiang,et al.  Multiscale Locality and Rank Preservation for Robust Feature Matching of Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[26]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Min Jiang,et al.  As-Aligned-As-Possible Image Stitching Based on Deviation-Corrected Warping With Global Similarity Constraints , 2019, IEEE Access.

[28]  Yu-Sheng Chen,et al.  Natural Image Stitching with the Global Similarity Prior , 2016, ECCV.

[29]  Lizhe Wang,et al.  SuperPCA: A Superpixelwise PCA Approach for Unsupervised Feature Extraction of Hyperspectral Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Jon Atli Benediktsson,et al.  Recent Advances in Techniques for Hyperspectral Image Processing , 2009 .

[31]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[32]  Seyed Mostafa Mousavi Kahaki,et al.  Accurate registration of in vivo time-lapse images , 2019, Medical Imaging: Image Processing.

[33]  Michael S. Brown,et al.  Constructing image panoramas using dual-homography warping , 2011, CVPR 2011.