Non-linear generative embeddings for kernels on latent variable models

Generative embeddings use generative probabilistic models to project objects into a vectorial space of reduced dimensionality - where the so-called generative kernels can be defined. Some of these approaches employ generative models on latent variables to project objects into a feature space where the dimensions are related to the latent variables. Here, we propose to enhance the discriminative power of such spaces by performing a non-linear mapping of space dimensions leading to the formulation of novel generative kernels. In this paper, we investigate one possible non-linear mapping, based on a powering operation, able to equilibrate the contributions of each latent variable of the model, thus augmenting the entropy of the latent variables vectors. The validity of the idea has been shown in the case of two generative kernels, which have been evaluated with tests on shape recognition and gesture classification, with really satisfying results that outperform state-of-the-art methods.

[1]  Sergios Theodoridis,et al.  Pattern Recognition , 1998, IEEE Trans. Neural Networks.

[2]  Horst Bunke,et al.  Edit distance-based kernel functions for structural pattern classification , 2006, Pattern Recognit..

[3]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[4]  N. D. Smith,et al.  Using Augmented Statistical Models and Score Spaces for Classification , 2003 .

[5]  Mark J. F. Gales,et al.  Speech Recognition using SVMs , 2001, NIPS.

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[8]  Mário A. T. Figueiredo,et al.  Similarity-based classification of sequences using hidden Markov models , 2004, Pattern Recognit..

[9]  Robert P. W. Duin,et al.  Component-based discriminative classification for hidden Markov models , 2009, Pattern Recognit..

[10]  Tony Jebara,et al.  Probability Product Kernels , 2004, J. Mach. Learn. Res..

[11]  Mohammed Waleed Kadous,et al.  Learning Comprehensible Descriptions of Multivariate Time Series , 1999, ICML.

[12]  Shigeaki Watanabe,et al.  Subspace method to pattern recognition , 1973 .

[13]  Manuele Bicego,et al.  Investigating hidden Markov models' capabilities in 2D shape classification , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Gabriela Andreu,et al.  Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[15]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[16]  Alessandro Perina,et al.  A New Generative Feature Set Based on Entropy Distance for Discriminative Classification , 2009, ICIAP.

[17]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[18]  Manuele Bicego,et al.  2D Shape Classification Using Multifractional Brownian Motion , 2008, SSPR/SPR.

[19]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[20]  Kiyoshi Asai,et al.  Marginalized kernels for biological sequences , 2002, ISMB.

[21]  Hong Man,et al.  Face recognition based on multi-class mapping of Fisher scores , 2005, Pattern Recognit..

[22]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[23]  Francisco Casacuberta,et al.  Cyclic Sequence Alignments: Approximate Versus Optimal Techniques , 2002, Int. J. Pattern Recognit. Artif. Intell..