Discriminative k-metrics

The k q-flats algorithm is a generalization of the popular k-means algorithm where q dimensional best fit affine sets replace centroids as the cluster prototypes. In this work, a modification of the k q-flats framework for pattern classification is introduced. The basic idea is to replace the original reconstruction only energy, which is optimized to obtain the k affine spaces, by a new energy that incorporates discriminative terms. This way, the actual classification task is introduced as part of the design and optimization. The presentation of the proposed framework is complemented with experimental results, showing that the method is computationally very efficient and gives excellent results on standard supervised learning benchmarks.

[1]  Cordelia Schmid,et al.  Vector Quantizing Feature Space with a Regular Lattice , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Michael B. Wakin,et al.  The geometry of low-dimensional signal models , 2007 .

[3]  Paul S. Bradley,et al.  k-Plane Clustering , 2000, J. Glob. Optim..

[4]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[5]  Bernhard Schölkopf,et al.  Introduction to Semi-Supervised Learning , 2006, Semi-Supervised Learning.

[6]  Dario Maio,et al.  Multispace KL for Pattern Representation and Classification , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Joel A. Tropp,et al.  Topics in sparse approximation , 2004 .

[8]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[9]  P. Tseng Nearest q-Flat to m Points , 2000 .

[10]  Nanda Kambhatla,et al.  Fast Non-Linear Dimension Reduction , 1993, NIPS.

[11]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[12]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[13]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[14]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[15]  Cordelia Schmid,et al.  Combining Regions and Patches for Object Class Localization , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[16]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.