Keypoint Detection in RGBD Images Based on an Anisotropic Scale Space

The increasing availability of texture+depth (RGBD) content has recently motivated research toward the design of image features able to employ the additional geometrical information provided by depth. Indeed, such features are supposed to provide higher robustness than conventional 2D features in the presence of large changes of camera viewpoint. In this paper, we consider the first stage of RGBD image matching, i.e., keypoint detection. In order to obtain viewpoint-covariant keypoints, we design a filtering process, which approximates a diffusion process along the surfaces of the scene, by means of the information provided by depth. Next, we employ this multiscale representation to find keypoints through a multiscale keypoint detector. The keypoints obtained by the proposed detector provide substantially higher stability to viewpoint changes than alternative 2D and RGBD feature extraction approaches, both in terms of repeatability and image classification accuracy. Furthermore, the proposed detector can be efficiently implemented on a GPU.

[1]  Vicent Caselles,et al.  Multiscale Analysis for Images on Riemannian Manifolds , 2014, SIAM J. Imaging Sci..

[2]  Matthew A. Brown,et al.  Invariant Features from Interest Point Groups , 2002, BMVC.

[3]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[4]  Chong-Wah Ngo,et al.  Representations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study , 2010, IEEE Transactions on Multimedia.

[5]  Pedro Arias,et al.  Metrological evaluation of Microsoft Kinect and Asus Xtion sensors , 2013 .

[6]  Qi Tian,et al.  Uniting Keypoints: Local Visual Information Fusion for Large-Scale Image Search , 2015, IEEE Transactions on Multimedia.

[7]  Alexander M. Bronstein,et al.  Photometric Heat Kernel Signatures , 2011, SSVM.

[8]  Marco Tagliasacchi,et al.  Cooperative image analysis in visual sensor networks , 2015, Ad Hoc Networks.

[9]  Adnan Yazici,et al.  Towards Effective Image Classification Using Class-Specific Codebooks and Distinctive Local Features , 2015, IEEE Transactions on Multimedia.

[10]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[11]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[12]  Wei Li,et al.  Fully affine invariant SURF for image matching , 2012, Neurocomputing.

[13]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[14]  Hongjian You,et al.  BFSIFT: A Novel Method to Find Feature Matches for SAR Image Registration , 2012, IEEE Geoscience and Remote Sensing Letters.

[15]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[16]  Lars Bretzner,et al.  Real-Time Scale Selection in Hybrid Multi-scale Representations , 2003, Scale-Space.

[17]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[18]  Cordelia Schmid,et al.  An Affine Invariant Interest Point Detector , 2002, ECCV.

[19]  Giuseppe Valenzise,et al.  Improving distinctiveness of brisk features using depth maps , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[20]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[21]  Reinhard Koch,et al.  Perspectively Invariant Normal Features , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Pietro Perona,et al.  Evaluation of Features Detectors and Descriptors Based on 3D Objects , 2005, ICCV.

[23]  Ivana Tosic,et al.  3D keypoint detection by light field scale-depth space analysis , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[24]  Jan-Michael Frahm,et al.  3D model matching with Viewpoint-Invariant Patches (VIP) , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Giuseppe Valenzise,et al.  A scale space for texture+depth images based on a discrete laplacian operator , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[26]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[27]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[28]  David Suter,et al.  Feature Detection with an Improved Anisotropic Filter , 2006, ACCV.

[29]  Alan C. Bovik,et al.  Natural scene statistics of color and range , 2011, 2011 18th IEEE International Conference on Image Processing.

[30]  G. Sapiro,et al.  Geometric partial differential equations and image analysis [Book Reviews] , 2001, IEEE Transactions on Medical Imaging.

[31]  Federico Tombari,et al.  A combined texture-shape descriptor for enhanced 3D feature matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Jan-Michael Frahm,et al.  Comparative Evaluation of Binary Features , 2012, ECCV.

[33]  Le Xiao,et al.  SIPF: Scale invariant point feature for 3D point clouds , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[34]  J. Koenderink.  The structure of images , 2004, Biological Cybernetics.

[35]  Mario Fernando Montenegro Campos,et al.  On the development of a robust, fast and lightweight keypoint descriptor , 2013, Neurocomputing.

[36]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Giuseppe Valenzise,et al.  Local visual features extraction from texture+depth content based on depth image analysis , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[38]  Wolfram Burgard,et al.  Point feature extraction on 3D range scans taking into account object boundaries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[39]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[40]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[42]  Horst Bischof,et al.  A novel performance evaluation method of local detectors on non-planar scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[43]  Alberto Del Bimbo,et al.  The Mesh-LBP: A Framework for Extracting Local Binary Patterns From Discrete Manifolds , 2015, IEEE Transactions on Image Processing.

[44]  Alan C. Bovik,et al.  Color and Depth Priors in Natural Images , 2013, IEEE Transactions on Image Processing.

[45]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[46]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[47]  Q. M. Jonathan Wu,et al.  A comparative experimental study of image feature detectors and descriptors , 2015, Machine Vision and Applications.

[48]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[49]  João Ascenso,et al.  Evaluation of low-complexity visual feature detectors and descriptors , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[50]  R. Horaud,et al.  Surface feature detection and description with applications to mesh matching , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  J. Paul Siebert,et al.  Local feature extraction and matching on range images: 2.5D SIFT , 2009, Comput. Vis. Image Underst..