A lower bound for generalized median based consensus learning using kernel-induced distance functions

Abstract Computing a consensus object from a set of given objects is a core problem in machine learning and pattern recognition. One commonly used approach is to formulate it as an optimization problem using the generalized median. However, in many domains the construction of a median object is NP -hard, requiring approximate solutions instead. In these cases a lower bound is helpful to assess the quality of an approximate median without the need of a ground-truth median result. In this work we introduce a domain-independent lower bound formulation for the generalized median using kernel-induced distances. As kernel functions induce a scalar product in a high-dimensional vector space, they can be used to construct new nonlinear distance functions in the original space with desirable properties. Using the properties of the kernel functions at the basis of the kernel-induced distances, this kernel-based lower bound formulation is shown to be considerably tighter than existing lower bounds on a number of datasets in four different domains, while requiring a lower computational time.

[1]  Xiaoyi Jiang,et al.  Evolutionary Weighted Mean Based Framework for Generalized Median Computation with Application to Strings , 2012, SSPR/SPR.

[2]  Mahantapas Kundu,et al.  The journey of graph kernels through two decades , 2018, Comput. Sci. Rev..

[3]  Xiaoyi Jiang,et al.  A hybrid reproducing graph kernel based on information entropy , 2018, Pattern Recognit..

[4]  Suresh Venkatasubramanian,et al.  Comparing distributions and shapes using the kernel distance , 2010, SoCG '11.

[5]  Daoqiang Zhang,et al.  Improving the Robustness of ‘Online Agglomerative Clustering Method’ Based on Kernel-Induce Distance Measures , 2005, Neural Processing Letters.

[6]  Andreas Weinmann,et al.  Fast Median Filtering for Phase or Orientation Data , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Michalis Vazirgiannis,et al.  GraKeL: A Graph Kernel Library in Python , 2018, J. Mach. Learn. Res..

[8]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[9]  Francesca Odone,et al.  Histogram intersection kernel for image classification , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[10]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[11]  Jinhui Xu,et al.  k-Prototype Learning for 3D Rigid Structures , 2013, NIPS.

[12]  Thomas Gärtner,et al.  Kernels and Distances for Structured Data , 2004, Machine Learning.

[13]  Xiaoyi Jiang,et al.  Generalized median string computation by means of string embedding in vector spaces , 2012, Pattern Recognit. Lett..

[14]  Nir Ailon,et al.  Aggregating inconsistent information: Ranking and clustering , 2008 .

[15]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[16]  Horst Bunke,et al.  Optimal Lower Bound for Generalized Median Problems in Metric Space , 2002, SSPR/SPR.

[17]  Jean-Philippe Vert,et al.  The Kendall and Mallows Kernels for Permutations , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Xiaoyi Jiang,et al.  Ensemble clustering by means of clustering embedding in vector spaces , 2014, Pattern Recognit..

[19]  Julia Couto,et al.  Kernel K-Means for Categorical Data , 2005, IDA.

[20]  Vincent Conitzer,et al.  Improved Bounds for Computing Kemeny Rankings , 2006, AAAI.

[21]  Jiming Peng,et al.  Generalized median graphs and applications , 2009, J. Comb. Optim..

[22]  Martin Bader,et al.  The transposition median problem is NP-complete , 2011, Theor. Comput. Sci..

[23]  Daoqiang Zhang,et al.  Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Y. Lim,et al.  Matrix power means and the Karcher mean , 2012 .

[25]  Bernard M. E. Moret,et al.  Finding an Optimal Inversion Median: Experimental Results , 2001, WABI.

[26]  Xiaoyi Jiang,et al.  Image Segmentation Fusion Using General Ensemble Clustering Methods , 2010, ACCV.

[27]  Xiaoyi Jiang,et al.  Distance-Preserving Vector Space Embedding for Consensus Learning , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Xiaoyi Jiang,et al.  Exploring the Performance Limit of Cluster Ensemble Techniques , 2010, SSPR/SPR.

[29]  Vladimir Filkov,et al.  Consensus Clustering Algorithms: Comparison and Refinement , 2008, ALENEX.

[30]  Chang-Dong Wang,et al.  Ensemble clustering using factor graph , 2016, Pattern Recognit..

[31]  Ernest Valveny,et al.  Generalized median graph computation by means of graph embedding in vector spaces , 2010, Pattern Recognit..

[32]  Xiaoyi Jiang,et al.  Comparison of Methods for Hyperspherical Data Averaging and Parameter Estimation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[33]  Venu Madhav Govindu,et al.  Robust Relative Rotation Averaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Xiaoyi Jiang,et al.  Consensus Learning for Sequence Data , 2018 .

[35]  Huilin Xiong,et al.  Kernel-based distance metric learning for microarray data classification , 2006, BMC Bioinformatics.

[36]  Baba C. Vemuri,et al.  Multiple Atlas Construction From A Heterogeneous Brain MR Image Collection , 2013, IEEE Transactions on Medical Imaging.

[37]  Bernard De Baets,et al.  Kernel Distance Metric Learning Using Pairwise Constraints for Person Re-Identification , 2019, IEEE Transactions on Image Processing.

[38]  Alberto Muñoz,et al.  Extending the SOM Algorithm to Non-Euclidean Distances via the Kernel Trick , 2004, ICONIP.

[39]  Juan Ramón Rico-Juan,et al.  A new iterative algorithm for computing a quality approximate median of strings based on edit operations , 2014, Pattern Recognit. Lett..

[40]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[41]  Frank Plastria,et al.  On the point for which the sum of the distances to n given points is minimum , 2009, Ann. Oper. Res..

[42]  E. Granum,et al.  Quantitative analysis of 6985 digitized trypsin G ‐banded human metaphase chromosomes , 1980, Clinical genetics.

[43]  Nanning Zheng,et al.  Exact solution to median surface problem using 3D graph search and application to parameter space exploration , 2015, Pattern Recognit..

[44]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[45]  Jinhui Xu,et al.  Finding Median Point-Set Using Earth Mover's Distance , 2014, AAAI.

[46]  Sandro Vega-Pons,et al.  A Survey of Clustering Ensemble Algorithms , 2011, Int. J. Pattern Recognit. Artif. Intell..

[47]  Tossapon Boongoen,et al.  Cluster ensembles: A survey of approaches with recent extensions and applications , 2018, Comput. Sci. Rev..

[48]  Hans-Peter Kriegel,et al.  Shortest-path kernels on graphs , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[49]  Francisco Casacuberta,et al.  Topology of Strings: Median String is NP-Complete , 1999, Theor. Comput. Sci..

[50]  Dan Gusfield,et al.  Algorithms on Strings, Trees, and Sequences - Computer Science and Computational Biology , 1997 .

[51]  Thomas Gärtner,et al.  A survey of kernels for structured data , 2003, SKDD.

[52]  Anthony Wirth,et al.  Are approximation algorithms for consensus clustering worthwhile? , 2007, SDM.

[53]  Alain Denise,et al.  Using Medians to Generate Consensus Rankings for Biological Data , 2011, SSDBM.

[54]  Benjamin Berkels,et al.  An SL(2) Invariant Shape Median , 2010, Journal of Mathematical Imaging and Vision.

[55]  Alain Denise,et al.  Rank aggregation with ties: Experiments and Analysis , 2015, Proc. VLDB Endow..

[56]  Francesco Masulli,et al.  A survey of kernel and spectral methods for clustering , 2008, Pattern Recognit..

[57]  Horst Bunke,et al.  On Median Graphs: Properties, Algorithms, and Applications , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Cong Li,et al.  Kernel-based distance metric learning in the output space , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[59]  Jacob Goldberger,et al.  Ensemble Segmentation Using Efficient Integer Linear Programming , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.