Probabilistic Zero-shot Classification with Semantic Rankings

In this paper we propose a non-metric ranking-based representation of semantic similarity that allows natural aggregation of semantic information from multiple heterogeneous sources. We apply the ranking-based representation to zero-shot learning problems, and present deterministic and probabilistic zero-shot classifiers which can be built from pre-trained classifiers without retraining. We demonstrate their the advantages on two large real-world image datasets. In particular, we show that aggregating different sources of semantic information, including crowd-sourcing, leads to more accurate classification.

[1]  Andrew Y. Ng,et al.  Improving Word Representations via Global Context and Multiple Word Prototypes , 2012, ACL.

[2]  Andrew Y. Ng,et al.  Zero-Shot Learning Through Cross-Modal Transfer , 2013, NIPS.

[3]  Cordelia Schmid,et al.  Label-Embedding for Attribute-Based Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Marc'Aurelio Ranzato,et al.  DeViSE: A Deep Visual-Semantic Embedding Model , 2013, NIPS.

[5]  Joseph S. Verducci,et al.  Probability models on rankings. , 1991 .

[6]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[7]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[8]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[9]  C. L. Mallows.  NON-NULL RANKING MODELS. I , 1957 .

[10]  Ayala Cohen,et al.  On a Model for Concordance between Judges , 1978 .

[11]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[12]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[13]  D. Hunter.  MM algorithms for generalized Bradley-Terry models , 2003 .

[14]  Patricia Rose Gomes de Melo Viol Martins,et al.  MATHEMATICS WITHOUT NUMBERS: AN INTRODUCTION TO THE STUDY OF LOGIC , 2015 .

[15]  D. Critchlow.  Metric Methods for Analyzing Partially Ranked Data , 1986 .

[16]  Charu C. Aggarwal,et al.  Towards cross-category knowledge propagation for learning visual concepts , 2011, CVPR 2011.

[17]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[18]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[19]  R. Plackett.  The Analysis of Permutations , 1975 .

[20]  Eyke Hüllermeier,et al.  Label Ranking Methods based on the Plackett-Luce Model , 2010, ICML.

[21]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[22]  Bernt Schiele,et al.  Evaluating knowledge transfer and zero-shot learning in a large-scale setting , 2011, CVPR 2011.

[23]  Bernt Schiele,et al.  What helps where – and why? Semantic relatedness for knowledge transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Thomas Deselaers,et al.  Visual and semantic similarity in ImageNet , 2011, CVPR 2011.

[25]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  M. Fligner,et al.  Distance Based Ranking Models , 1986 .

[27]  Jeff A. Bilmes,et al.  Consensus ranking under the exponential model , 2007, UAI.

[28]  M. Trick,et al.  Voting schemes for which it can be difficult to tell who won the election , 1989 .

[29]  Gabriela Csurka,et al.  Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost , 2012, ECCV.

[30]  Cees Snoek,et al.  COSTA: Co-Occurrence Statistics for Zero-Shot Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Harry Joe,et al.  On the Babington Smith Class of Models for Rankings , 1993 .

[32]  Geoffrey E. Hinton,et al.  Zero-shot Learning with Semantic Output Codes , 2009, NIPS.