Quaternion Harris For Multispectral Keypoint Detection

We present a new keypoint detection method that generalizes Harris corners for multispectral images by considering the input as a quaternionic matrix. Standard keypoint detectors run on scalar-valued inputs, neglecting input multimodality and potentially missing highly distinctive features. The proposed detector uses information from all channel inputs by defining a quaternionic autocorrelation matrix that possesses quaternionic eigenvectors and real eigenvalues, for the computation of which channel cross-correlations are also taken into account. We have tested the proposed detector on a variety of multispectral images (color, near-infrared), where we have validated its usefulness.

[1]  Michael G. Strintzis,et al.  3D object articulation and motion estimation in model-based stereoscopic videoconference image sequence analysis and coding , 1999, Signal Process. Image Commun..

[2]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[3]  Dmytro Mishkin,et al.  Kornia: an Open Source Differentiable Computer Vision Library for PyTorch , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Fuzhen Zhang.  Quaternions and matrices of quaternions , 1997 .

[5]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .

[6]  Stephen J. Sangwine,et al.  Hypercomplex Fourier Transforms of Color Images , 2001, IEEE Transactions on Image Processing.

[7]  Krystian Mikolajczyk,et al.  Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Jiri Matas,et al.  Working hard to know your neighbor's margins: Local descriptor learning loss , 2017, NIPS.

[9]  Kin-Man Lam,et al.  Salient object detection using octonion with Bayesian inference , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[10]  Xiang Li,et al.  A quaternion gradient operator for color image edge detection , 2013, 2013 IEEE International Conference on Image Processing.

[11]  Nicolas Le Bihan,et al.  Singular value decomposition of quaternion matrices: a new tool for vector-sensor signal processing , 2004, Signal Process..

[12]  Isao Yamada,et al.  Hypercomplex Tensor Completion with Cayley-Dickson Singular Value Decomposition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Martin Humenberger,et al.  R2D2: Reliable and Repeatable Detector and Descriptor , 2019, NeurIPS.

[14]  Torsten Sattler,et al.  Comparative Evaluation of Hand-Crafted and Learned Local Features , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sabine Süsstrunk,et al.  Multispectral interest points for RGB-NIR image registration , 2011, 2011 18th IEEE International Conference on Image Processing.

[16]  Nikolas P. Galatsanos,et al.  Majorization-minimization mixture model determination in image segmentation , 2011, CVPR 2011.

[17]  Giorgos Sfikas,et al.  A Deep Neural Network-Based Method For The Detection And Accurate Thermography Statistics Estimation Of Aerially Surveyed Structures , 2019 .

[18]  Christophoros Nikou,et al.  Recovery of polarimetric Stokes images by spatial mixture models. , 2011, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  Michael G. Strintzis,et al.  Robust image watermarking in the subband or discrete cosine transform domain , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[20]  Sabine Süsstrunk,et al.  Multi-spectral SIFT for scene category recognition , 2011, CVPR 2011.