VPALG: Paper-publication Prediction with Graph Neural Networks

Paper-publication venue prediction aims to predict candidate publication venues that effectively suit given submissions. This technology is developing rapidly with the popularity of machine learning models. However, most previous methods ignore the structure information of papers, while modeling them with graphs can naturally solve this drawback. Meanwhile, they either use hand-crafted or bag-of-word features to represent the papers, ignoring the ones that involve high-level semantics. Moreover, existing methods assume that the venue where a paper is published as a correct venue for the data annotation, which is unrealistic. One paper can be relevant to many venues. In this paper, we attempt to address these problems above and develop a novel prediction model, namelyVenue Prediction with Abstract-Level Graph (Vpalg xspace), which can serve as an effective decision-making tool for venue selections. Specifically, to achieve more discriminative paper abstract representations, we construct each abstract as a semantic graph and perform a dual attention message passing neural network for representation learning. Then, the proposed model can be trained over the learned abstract representations with their labels and generalized via self-training. Empirically, we employ the PubMed dataset and further collect two new datasets from the top journals and conferences in computer science. Experimental results indicate the superior performance of Vpalg xspace, consistently outperforming the existing baseline methods.

[1]  Yuji Matsumoto,et al.  Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach , 2017, ArXiv.

[2]  Feng Xia,et al.  PAVE: Personalized Academic Venue recommendation Exploiting co-publication networks , 2018, J. Netw. Comput. Appl..

[3]  Sukomal Pal,et al.  CNAVER: A Content and Network-based Academic VEnue Recommender system , 2020, Knowl. Based Syst..

[4]  Nikolaos G. Bourbakis,et al.  Graph-Based Methods for Natural Language Processing and Understanding—A Survey and Analysis , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Philip S. Yu,et al.  Hierarchical Bi-Directional Self-Attention Networks for Paper Review Rating Recommendation , 2020, COLING.

[6]  Richard Furuta,et al.  Recommendation of Scholarly Venues Based on Dynamic User Interests , 2016, J. Informetrics.

[7]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  CHENGXIANG ZHAI,et al.  A study of smoothing methods for language models applied to information retrieval , 2004, TOIS.

[10]  Bela Gipp,et al.  Research-paper recommender systems: a literature survey , 2015, International Journal on Digital Libraries.

[11]  Thorsten Joachims,et al.  Evaluation methods for unsupervised word embeddings , 2015, EMNLP.

[12]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[13]  E. Valuations,et al.  A R EVIEW ON E VALUATION M ETRICS F OR D ATA C LASSIFICATION E VALUATIONS , 2015 .

[14]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[15]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[16]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[17]  Quoc V. Le,et al.  Rethinking Pre-training and Self-training , 2020, NeurIPS.

[18]  Xuanjing Huang,et al.  Recurrent Neural Network for Text Classification with Multi-Task Learning , 2016, IJCAI.

[19]  Michalis Vazirgiannis,et al.  Message Passing Attention Networks for Document Understanding , 2019, AAAI.

[20]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[21]  Brian D. Davison,et al.  Venue Recommendation: Submitting Your Paper with Style , 2012, 2012 11th International Conference on Machine Learning and Applications.

[22]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[23]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[24]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[25]  W. Bruce Croft,et al.  A Language Modeling Approach to Information Retrieval , 1998, SIGIR Forum.

[26]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[27]  Zhendong Niu,et al.  Recommending scientific paper via heterogeneous knowledge embedding based attentive recurrent neural networks , 2021, Knowl. Based Syst..

[28]  Eric Medvet,et al.  Publication Venue Recommendation Based on Paper Abstract , 2014, 2014 IEEE 26th International Conference on Tools with Artificial Intelligence.

[29]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[30]  Nanning Zheng,et al.  Transductive Semi-Supervised Deep Learning Using Min-Max Features , 2018, ECCV.

[31]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[32]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[33]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[34]  Sukomal Pal,et al.  HASVRec: A modularized Hierarchical Attention-based Scholarly Venue Recommender system , 2020, Knowl. Based Syst..

[35]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[36]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Yi Zhu,et al.  The Deep Learning–Based Recommender System “Pubmender” for Choosing a Biomedical Publication Venue: Development and Validation Study , 2019, Journal of medical Internet research.

[38]  Feng Xia,et al.  AVER: Random Walk Based Academic Venue Recommendation , 2015, WWW.

[39]  Bowen Zhou,et al.  A Structured Self-attentive Sentence Embedding , 2017, ICLR.

[40]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[41]  Donghui Wang,et al.  A content-based recommender system for computer science publications , 2018, Knowl. Based Syst..

[42]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[43]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .