A new dictionary-based positive and unlabeled learning method

Positive and unlabeled learning (PU learning) is designed to solve the problem that we only utilize the labeled positive examples and the unlabeled examples to train a classifier. A variety of methods have been proposed to solve this problem by incorporating unlabeled examples into learning. However, many methods treat the original features as input in the training stage and then build the classifier. In this paper, by use of two-step strategy, a novel method with dictionary learning is proposed for PU learning, which is briefly called PUDL. The proposed method is done in two steps. Firstly, we extract reliable negative examples from unlabeled examples to form the negative class. We then utilize dictionary learning to construct the feature representation which can learn new features for the original input. Secondly, we propose RankSVM-based (Ranking Support Vector Machine) model to incorporate the positive class, extracted negative class and similarity weights into learning so as to improve performance of the classifier. The Lagrange multiplier method is applied to convert the original model into its dual form. In addition, we put forward an interactive optimization framework to optimize the proposed objective model and obtain the classifier. Finally, we verify the performance of PUDL via experiment and the results demostrate that PUDL performs better than state-of-the-art PU learning methods.

[1]  Luc Van Gool,et al.  Latent Dictionary Learning for Sparse Representation Based Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Lu Liu,et al.  Building text classifiers using positive, unlabeled and ‘outdated’ examples , 2016, Concurr. Comput. Pract. Exp..

[3]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[4]  François Denis PAC Learning from Positive Statistical Queries , 1998, ALT.

[5]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[6]  Yuan-Hai Shao,et al.  Laplacian unit-hyperplane learning from positive and unlabeled examples , 2015, Inf. Sci..

[7]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[8]  Fan Li,et al.  Ranking specialization for web search: a divide-and-conquer approach by using topical RankSVM , 2010, WWW '10.

[9]  Wenkai Li,et al.  A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Ning Ye,et al.  Boundary detection and sample reduction for one-class Support Vector Machines , 2014, Neurocomputing.

[11]  Minh-Tien Nguyen,et al.  Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network , 2017, ArXiv.

[12]  Gang Niu,et al.  Positive-Unlabeled Learning with Non-Negative Risk Estimator , 2017, NIPS.

[13]  Rama Chellappa,et al.  Analysis sparse coding models for image-based classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[14]  Yang Wang,et al.  Effectively Identifying Compound-Protein Interactions by Learning from Positive and Unlabeled Examples. , 2018, IEEE/ACM transactions on computational biology and bioinformatics.

[15]  Ping Li,et al.  Cross-Domain Person Reidentification Using Domain Adaptation Ranking SVMs , 2015, IEEE Transactions on Image Processing.

[16]  Ling Jing,et al.  Global and local learning from positive and unlabeled examples , 2017, Applied Intelligence.

[17]  Ian H. Witten,et al.  One-Class Classification by Combining Density and Class Probability Estimation , 2008, ECML/PKDD.

[18]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[19]  Xiaoli Li,et al.  Ensemble Positive Unlabeled Learning for Disease Gene Identification , 2014, PloS one.

[20]  Yoram Bresler,et al.  $\ell_{0}$ Sparsifying Transform Learning With Efficient Optimal Updates and Convergence Guarantees , 2015, IEEE Transactions on Signal Processing.

[21]  Lars Schmidt-Thieme,et al.  Cost-sensitive learning methods for imbalanced data , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[22]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[23]  Hamidreza Saligheh Rad,et al.  Accurate classification of brain gliomas by discriminate dictionary learning based on projective dictionary pair learning of proton magnetic resonance spectra , 2017, Magnetic resonance in chemistry : MRC.

[24]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[25]  Kotagiri Ramamohanarao,et al.  Learning with Bounded Instance- and Label-dependent Label Noise , 2017, ICML.

[26]  Chien-Yao Wang,et al.  Locality-preserving K-SVD Based Joint Dictionary and Classifier Learning for Object Recognition , 2016, ACM Multimedia.

[27]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[28]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[29]  Hui Liu,et al.  Effectively Identifying Compound-Protein Interactions by Learning from Positive and Unlabeled Examples , 2018, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[30]  Bing Liu,et al.  Learning with Positive and Unlabeled Examples Using Weighted Logistic Regression , 2003, ICML.

[31]  Jianfei Cai,et al.  Robust surface reconstruction via dictionary learning , 2014, ACM Trans. Graph..

[32]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[33]  Shuicheng Yan,et al.  Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Tommy W. S. Chow,et al.  Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation , 2016, IEEE Transactions on Signal Processing.

[35]  Philip S. Yu,et al.  Building text classifiers using positive and unlabeled examples , 2003, Third IEEE International Conference on Data Mining.

[36]  S. Sathiya Keerthi,et al.  A pairwise ranking based approach to learning with positive and unlabeled examples , 2011, CIKM '11.

[37]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[38]  Pedro Larrañaga,et al.  Learning Bayesian classifiers from positive and unlabeled examples , 2007, Pattern Recognit. Lett..

[39]  Yi Pan,et al.  Predicting drug-target interaction using positive-unlabeled learning , 2016, Neurocomputing.

[40]  François Laviolette,et al.  Accelerated Robust Point Cloud Registration in Natural Environments through Positive and Unlabeled Learning , 2013, IJCAI.

[41]  Adel M. Alimi,et al.  Natural Language Morphology Integration in Off-Line Arabic Optical Text Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Qiang Yang,et al.  Learning with Positive and Unlabeled Examples Using Topic-Sensitive PLSA , 2010, IEEE Transactions on Knowledge and Data Engineering.

[43]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[44]  Junsong Yuan,et al.  Positive and Unlabeled Learning for Anomaly Detection with Multi-features , 2017, ACM Multimedia.

[45]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[47]  Yoram Bresler,et al.  Learning overcomplete sparsifying transforms with block cosparsity , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[48]  Hamid R. Rabiee,et al.  Spatial-Aware Dictionary Learning for Hyperspectral Image Classification , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Philip S. Yu,et al.  Text classification without negative examples revisit , 2006, IEEE Transactions on Knowledge and Data Engineering.

[50]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[52]  Jiawei Han,et al.  PEBL: Web page classification without negative examples , 2004, IEEE Transactions on Knowledge and Data Engineering.

[53]  Malik Yousef,et al.  One-Class SVMs for Document Classification , 2002, J. Mach. Learn. Res..

[54]  David Zhang,et al.  Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification , 2014, International Journal of Computer Vision.

[55]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[56]  Dacheng Tao,et al.  Learning with Biased Complementary Labels , 2017, ECCV.

[57]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Charles Elkan,et al.  Learning classifiers from only positive and unlabeled data , 2008, KDD.

[59]  De-Shuang Huang,et al.  Multi-task ranking SVM for image cosegmentation , 2017, Neurocomputing.