Kernelizing Sorting, Permutation, and Alignment for Minimum Volume PCA

We propose an algorithm for permuting or sorting multiple sets (or bags) of objects such that they can ultimately be represented efficiently using kernel principal component analysis. This framework generalizes sorting from scalars to arbitrary inputs since all computations involve inner products which can be done in Hilbert space and kernelized. The cost function on the permutations or orderings emerges from a maximum likelihood Gaussian solution which approximately minimizes the volume data occupies in Hilbert space. This ensures that few kernel principal components are necessary to capture the variation of the sets or bags. Both global and almost-global iterative solutions are provided in terms of iterative algorithms by interleaving variational bounding (on quadratic assignment problems) with a Kuhn-Munkres algorithm (for solving linear assignment problems).

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  Tony Jebara.  Convex Invarian e Learning , 2003 .

[3]  Yann LeCun,et al.  Transformation Invariance in Pattern Recognition-Tangent Distance and Tangent Propagation , 1996, Neural Networks: Tricks of the Trade.

[4]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[5]  Bernard Victorri,et al.  Transformation invariance in pattern recognition: Tangent distance and propagation , 2000 .

[6]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[7]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[8]  Eric Mjolsness,et al.  New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence , 1998, NIPS.

[9]  Panos M. Pardalos,et al.  Lower bounds for the quadratic assignment problem , 1994, Ann. Oper. Res..

[10]  James B. Orlin,et al.  QuickMatch--a very fast algorithm for the assignment problem , 1993 .

[11]  Sridevi Parise,et al.  Unsupervised Learning with Permuted Data , 2003, ICML.

[12]  Andrew V. Goldberg,et al.  An efficient cost scaling algorithm for the assignment problem , 1995, Math. Program..

[13]  Tony Jebara,et al.  Images as bags of pixels , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Tony Jebara,et al.  A Kernel Between Sets of Vectors , 2003, ICML.

[15]  Joshua B. Tenenbaum,et al.  Separating Style and Content with Bilinear Models , 2000, Neural Computation.

[16]  Tony Jebara.  Convex Invariance Learning , 2003, AISTATS.

[17]  Alan L. Yuille,et al.  The invisible hand algorithm: Solving the assignment problem with statistical physics , 1994, Neural Networks.