Overlapping area hyperspheres for kernel-based similarity method

Measuring similarity between sets of objects is a key step in a wide areas of machine learning. Popular examples include general classification framework and numerous applications in computer vision. In this paper, we propose a kernel-based similarity method which is inspired from an interesting biological behavior of trees and induced mathematically by formulating it as a quadratic optimization problem in a reproducing kernel Hilbert space (RKHS). The proposed method is compared to the maximum mean discrepancy, a recent and challenging kernel similarity method. We conduct and present several numerical experiments on synthetic data as well as real-word image data. The proposed method yields favorable performances in terms of classification performances in the context of supervised classification tasks on the challenging Caltech101 dataset and other datasets such as USPS and ETH80. Furthermore, the efficiency of the proposed method in the context of image segmentation through unsupervised clustering of superpixels has been also asserted.

[1]  Daphna Weinshall,et al.  Classification with Nonmetric Distances: Image Retrieval and Class Representation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Francisco Sandoval,et al.  From Natural to Artificial Neural Computation , 1995 .

[3]  Tony Jebara,et al.  A Kernel Between Sets of Vectors , 2003, ICML.

[4]  Mohamed S. Kamel,et al.  Unsupervised Visual Changepoint Detection Using Maximum Mean Discrepancy , 2013, ICIAR.

[5]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[6]  Todd K. Moon Similarity methods in signal processing , 1996, IEEE Trans. Signal Process..

[7]  Dinggang Shen,et al.  Matrix-Similarity Based Loss Function and Feature Selection for Alzheimer's Disease Diagnosis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  M. Hidayath Ansari,et al.  Learning from Spatial Overlap , 2011, AAAI.

[9]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[10]  Chiou-Shann Fuh,et al.  Multiple Kernel Learning for Dimensionality Reduction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Maurice Crosland,et al.  Gay-Lussac: Scientist and Bourgeois , 1978 .

[12]  Denis Teissandier,et al.  Minkowski sum of polytopes defined by their vertices , 2014, ArXiv.

[13]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Huilin Xiong,et al.  Kernel-based distance metric learning for microarray data classification , 2006, BMC Bioinformatics.

[15]  Eero P. Simoncelli,et al.  Differentiation of discrete multidimensional signals , 2004, IEEE Transactions on Image Processing.

[16]  Chih-Jen Lin,et al.  Training Support Vector Machines via SMO-Type Decomposition Methods , 2005, ALT.

[17]  Mikhail J. Atallah,et al.  A Linear Time Algorithm for the Hausdorff Distance Between Convex Polygons , 1983, Inf. Process. Lett..

[18]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[20]  George W. Furnas,et al.  Pictures of relevance: A geometric analysis of similarity measures , 1987, J. Am. Soc. Inf. Sci..

[21]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[22]  Bo Geng,et al.  DAML: Domain Adaptation Metric Learning , 2011, IEEE Transactions on Image Processing.

[23]  Joelle Pineau,et al.  Maximum Mean Discrepancy Imitation Learning , 2013, Robotics: Science and Systems.

[24]  S. Li Concise Formulas for the Area and Volume of a Hyperspherical Cap , 2011 .

[25]  Bernt Schiele,et al.  Analyzing contour and appearance based methods for object categorization , 2003, CVPR 2003.

[26]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[27]  Paul Bartha,et al.  By parallel reasoning : the construction and evaluation of analogical arguments , 2010 .

[28]  David W. Albrecht,et al.  Algorithms for the Computation of Reduced Convex Hulls , 2009, Australasian Conference on Artificial Intelligence.

[29]  Ghulam M. Bhat,et al.  Handwritten Digit Recognition using Slope Detail Features , 2014 .

[30]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[31]  Bing Wang,et al.  The Localized Consistency Principle for Image Matching under Non-uniform Illumination Variation and Affine Distortion , 2002, ECCV.

[32]  Robert P. W. Duin,et al.  AUTOMATIC PATTERN RECOGNITION BY SIMILARITY REPRESENTATIONS - A NOVEL APPROACH , 2001 .

[33]  Nicolai Petkov,et al.  From Natural to Artificial Neural Computation , 1995, Lecture Notes in Computer Science.

[34]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  Alexandre Bernardino,et al.  Topological Auto-Calibration of Central Imaging Sensors , 2013, IbPRIA.

[36]  Michael Diu,et al.  Image Analysis Applications of the Maximum Mean Discrepancy Distance Measure , 2013 .

[37]  R. Duin,et al.  Automatic pattern recognition by similarity representations , 2001 .

[38]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Xiaoqing Ding,et al.  Handwritten character recognition using gradient feature and quadratic classifier with multiple discrimination schemes , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[40]  Gian Luca Foresti,et al.  Trajectory-Based Anomalous Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[42]  Bernt Schiele,et al.  How Good are Local Features for Classes of Geometric Objects , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[43]  Alberto Del Bimbo,et al.  Local Pyramidal Descriptors for Image Recognition , 2014, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Bernhard Schölkopf,et al.  Learning from Distributions via Support Measure Machines , 2012, NIPS.

[45]  Alya Slimene,et al.  Kernel Maximum Mean Discrepancy for Region Merging Approach , 2013, CAIP.

[46]  K. Ball CONVEX BODIES: THE BRUNN–MINKOWSKI THEORY , 1994 .

[47]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[49]  Manuele Bicego,et al.  Dissimilarity-based representation for local parts , 2010, 2010 2nd International Workshop on Cognitive Information Processing.

[50]  A Cauchy-Schwarz type inequality for bilinear integrals on positive measures , 2005 .

[51]  J. Linnett The relation between potential energy and interatomic distance in some diatomic molecules , 1940 .

[52]  Ricardo Trumper,et al.  Teaching of Energy Issues: A Debate Proposal for a Global Reorientation , 2007 .

[53]  Hakan Cevikalp Semi-supervised Distance Metric Learning by Quadratic Programming , 2010, 2010 20th International Conference on Pattern Recognition.

[54]  Evimaria Terzi,et al.  A Framework for Evaluating the Smoothness of Data-Mining Results , 2012, ECML/PKDD.

[55]  Nicu Sebe,et al.  Similarity Matching in Computer Vision and Multimedia , 2008, Comput. Vis. Image Underst..

[56]  Marco Loog,et al.  On classification with bags, groups and sets , 2014, Pattern Recognit. Lett..

[57]  Brendan J. Frey,et al.  Non-metric affinity propagation for unsupervised image categorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[58]  Kenneth L. Clarkson,et al.  Optimal core-sets for balls , 2008, Comput. Geom..

[59]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[60]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[61]  Joseph S. B. Mitchell,et al.  Approximate minimum enclosing balls in high dimensions using core-sets , 2003, ACM J. Exp. Algorithmics.

[62]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[63]  A. Müller Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[64]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[65]  Miklós Simonovits,et al.  How to compute the volume in high dimension? , 2003, Math. Program..

[66]  Jitendra Malik,et al.  Geometric blur for template matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[67]  N̊ Thèse,et al.  Minkowski Sums of Polytopes : Combinatorics and Computation , 2007 .

[68]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[69]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[70]  Zheng Wang,et al.  Similarity Learning with Top-heavy Ranking Loss for Person Re-identification , 2016, IEEE Signal Processing Letters.

[71]  Ahmed M. Elgammal,et al.  Putting local features on a manifold , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[73]  Haizhou Li,et al.  An overview of text-independent speaker recognition: From features to supervectors , 2010, Speech Commun..

[74]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.