Density Estimators for Positive-Unlabeled Learning

Positive-Unlabeled (PU) learning works by considering a set of positive samples, and a (usually larger) set of unlabeled ones. This challenging setting requires algorithms to cleverly exploit dependencies hidden in the unlabeled data in order to build models able to accurately discriminate between positive and negative samples. We propose to exploit probabilistic generative models to characterize the distribution of the positive samples, and to label as reliable negative samples those that are in the lowest density regions with respect to the positive ones. The overall framework is flexible enough to be applied to many domains by leveraging tools provided by years of research from the probabilistic generative model community. Results on several benchmark datasets show the performance and flexibility of the proposed approach.

[1]  Chee Keong Kwoh,et al.  Positive-unlabeled learning for disease gene identification , 2012, Bioinform..

[2]  Philip S. Yu,et al.  Partially Supervised Classification of Text Documents , 2002, ICML.

[3]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[4]  Ruggero G. Pensa,et al.  Positive and unlabeled learning in categorical data , 2016, Neurocomputing.

[5]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[6]  Jing Xu,et al.  Intrusion Detection using Continuous Time Bayesian Networks , 2010, J. Artif. Intell. Res..

[7]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[8]  Pedro Larrañaga,et al.  Learning Bayesian classifiers from positive and unlabeled examples , 2007, Pattern Recognit. Lett..

[9]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[10]  Philip S. Yu,et al.  Positive and Unlabeled Learning for Graph Classification , 2011, 2011 IEEE 11th International Conference on Data Mining.

[11]  Charles Elkan,et al.  Learning classifiers from only positive and unlabeled data , 2008, KDD.

[12]  Philip S. Yu,et al.  Building text classifiers using positive and unlabeled examples , 2003, Third IEEE International Conference on Data Mining.

[13]  Ruggero G. Pensa,et al.  From Context to Distance: Learning Dissimilarity for Categorical Data Clustering , 2012, TKDD.

[14]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[15]  Pradeep Ravikumar,et al.  Mixed Graphical Models via Exponential Families , 2014, AISTATS.

[16]  Ian H. Witten,et al.  One-Class Classification by Combining Density and Class Probability Estimation , 2008, ECML/PKDD.

[17]  Teresa Maria Altomare Basile,et al.  Learning Bayesian Random Cutset Forests , 2015, ISMIS.

[18]  Masashi Sugiyama,et al.  Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching , 2012, ICML.

[19]  Pascal Vincent,et al.  Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives , 2012, ArXiv.

[20]  Floriana Esposito,et al.  Visualizing and understanding Sum-Product Networks , 2016, Machine Learning.

[21]  Bing Liu,et al.  Spotting Fake Reviews via Collective Positive-Unlabeled Learning , 2014, 2014 IEEE International Conference on Data Mining.

[22]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[23]  Kaizhu Huang,et al.  Biased support vector machine for relevance feedback in image retrieval , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[24]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[25]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[26]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[27]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[28]  Rémi Gilleron,et al.  Positive and Unlabeled Examples Help Learning , 1999, ALT.

[29]  Floriana Esposito,et al.  Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning , 2015, ECML/PKDD.

[30]  Qiang Yang,et al.  Learning with Positive and Unlabeled Examples Using Topic-Sensitive PLSA , 2010, IEEE Transactions on Knowledge and Data Engineering.

[31]  Qing Li,et al.  A Proposal for Statistical Outlier Detection in Relational Structures , 2014, StarAI@AAAI.

[32]  Floriana Esposito,et al.  Fast and Accurate Density Estimation with Extremely Randomized Cutset Networks , 2017, ECML/PKDD.

[33]  Floriana Esposito,et al.  Learning Accurate Cutset Networks by Exploiting Decomposability , 2015, AI*IA.

[34]  Daniel Lowd,et al.  The Libra toolkit for probabilistic models , 2015, J. Mach. Learn. Res..

[35]  Ivor W. Tsang,et al.  Multi-view Positive and Unlabeled Learning , 2012, ACML.