Keypoint descriptor fusion with Dempster-Shafer theory

Keypoint matching is the task of accurately finding the location of a scene point in two images. Many keypoint descriptors have been proposed in the literature aiming at providing robustness against scale, translation and rotation transformations, each having advantages and disadvantages. This paper proposes a novel approach to fuse the information from multiple keypoint descriptors using Dempster-Shafer theory of evidence 1], which has proven particularly efficient in combining sources of information providing incomplete, imprecise, biased, and conflictive knowledge. The matching results of each descriptor are transformed into an evidence distribution on which a confidence factor is computed making use of its entropy. Then, the evidence distributions are fused using Dempster-Shafer Theory (DST), considering its confidence. As a result of the fusion, a new evidence distribution that improves the result of the best descriptor is obtained. Our method has been tested with SIFT, SURF, ORB, BRISK and FREAK descriptors using all possible their combinations. Results on the Oxford keypoint dataset 2] show that the proposed approach obtains an improvement of up to 10 % compared to the best one (FREAK). Novel approach fusing information from multiple keypoint descriptors using the Dempster-Shafer Theory of evidence.Descriptors matches are transformed in evidence distributions assigning a confidence factor using Shannon's entropy.Results on the Oxford keypoint dataset show improvements of up to 10% compared to the best keypoint descriptor.

[1]  Thierry Denoeux,et al.  ECM: An evidential version of the fuzzy c , 2008, Pattern Recognit..

[2]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Guang-Zhong Yang,et al.  A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery , 2007, MICCAI.

[4]  Philippe Smets,et al.  The Combination of Evidence in the Transferable Belief Model , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Ioannis A. Kakadiaris,et al.  Feature fusion for facial landmark detection , 2014, Pattern Recognit..

[7]  Zou Yi,et al.  Multi-ultrasonic sensor fusion for autonomous mobile robots , 2000, SPIE Defense + Commercial Sensing.

[8]  Huijing Zhao,et al.  Multimodal information fusion for urban scene understanding , 2016, Machine Vision and Applications.

[9]  Véronique Berge-Cherfaoui,et al.  Optimal Object Association in the Dempster–Shafer Framework , 2014, IEEE Transactions on Cybernetics.

[10]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[11]  Jean-Philippe Thiran,et al.  Multisource clustering of remote sensing images with Entropy-based Dempster-Shafer fusion , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[12]  Thierry Denoeux,et al.  Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008, Artif. Intell..

[13]  Thierry Denoeux,et al.  Risk assessment based on weak information using belief functions: a case study in water treatment , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[14]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[15]  Philippe Smets,et al.  Belief functions: The disjunctive rule of combination and the generalized Bayesian theorem , 1993, Int. J. Approx. Reason..

[16]  Francisco José Madrid-Cuevas,et al.  Multi-camera people tracking using evidential filters , 2009, Int. J. Approx. Reason..

[17]  Zakia Hammal,et al.  Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model , 2007, Int. J. Approx. Reason..

[18]  Fabio Cuzzolin.  Geometry of relative plausibility and relative belief of singletons , 2010, Annals of Mathematics and Artificial Intelligence.

[19]  Philippe Smets,et al.  Computational aspects of the Mobius transformation , 1990, UAI.

[20]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[21]  Philippe Smets,et al.  Classification Using Belief Functions: Relationship Between Case-Based and Model-Based Approaches , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[23]  Dongjian He,et al.  A Multi-Descriptor, Multi-Nearest Neighbor Approach for Image Classification , 2010, ICIC.

[24]  Thierry Denoeux,et al.  Classifier fusion in the Dempster-Shafer framework using optimized t-norm based combination rules , 2011, Int. J. Approx. Reason..

[25]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Thierry Denoeux,et al.  EVCLUS: evidential clustering of proximity data , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[28]  Isabelle Bloch,et al.  Improving mine recognition through processing and Dempster-Shafer fusion of ground-penetrating radar data , 2003, Pattern Recognit..

[29]  Jie Yang,et al.  Sensor fusion using Dempster-Shafer theory [for context-aware HCI] , 2002, IMTC/2002. Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.00CH37276).

[30]  Yong Deng,et al.  A Ranking Procedure by Incomplete Pairwise Comparisons Using Information Entropy and Dempster-Shafer Evidence Theory , 2014, TheScientificWorldJournal.

[31]  Van-Nam Huynh,et al.  Adaptively entropy-based weighting classifiers in combination using Dempster-Shafer theory for word sense disambiguation , 2010, Comput. Speech Lang..

[32]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[33]  Fabio Cuzzolin.  The Intersection Probability and Its Properties , 2009, ECSQARU.

[34]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[35]  Wojciech Pieczynski,et al.  Multisensor triplet Markov chains and theory of evidence , 2007, Int. J. Approx. Reason..

[36]  Thierry Denoeux.  The cautious rule of combination for belief functions and some extensions , 2006, 2006 9th International Conference on Information Fusion.

[37]  Abdel-Ouahab Boudraa,et al.  Dempster-Shafer's Basic Probability Assignment Based on fuzzy Membership Functions , 2009, Progress in Computer Vision and Image Analysis.

[38]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[40]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[41]  Alessandro Saffiotti,et al.  The Transferable Belief Model , 1991, ECSQARU.

[42]  Isabelle Bloch,et al.  Defining belief functions using mathematical morphology - Application to image fusion under imprecision , 2008, Int. J. Approx. Reason..

[43]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[44]  T. Denœux.  Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008 .

[45]  P. K. Sa,et al.  Score level fusion of SIFT and SURF for iris , 2012, 2012 International Conference on Devices, Circuits and Systems (ICDCS).

[46]  Shamik Sural,et al.  Use of Dempster-Shafer Theory and Bayesian Inferencing for Fraud Detection in Mobile Communication Networks , 2007, ACISP.

[47]  Jiwen Lu,et al.  Robust Feature Set Matching for Partial Face Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[48]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[49]  Jie Yang,et al.  Sensor Fusion Using Dempster-Shafer Theory , 2002 .