Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search

This paper presents \textttConv-KNRM, a Convolutional Kernel-based Neural Ranking Model that models n-gram soft matches for ad-hoc search. Instead of exact matching query and document n-grams, \textttConv-KNRM uses Convolutional Neural Networks to represent n-grams of various lengths and soft matches them in a unified embedding space. The n-gram soft matches are then utilized by the kernel pooling and learning-to-rank layers to generate the final ranking score. \textttConv-KNRM can be learned end-to-end and fully optimized from user feedback. The learned model»s generalizability is investigated by testing how well it performs in a related domain with small amounts of training data. Experiments on English search logs, Chinese search logs, and TREC Web track tasks demonstrated consistent advantages of \textttConv-KNRM over prior neural IR methods and feature-based methods.

[1]  John D. Lafferty,et al.  Information retrieval as statistical translation , 1999, SIGIR '99.

[2]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[3]  Qun Liu,et al.  HHMM-based Chinese Lexical Analyzer ICTCLAS , 2003, SIGHAN.

[4]  Trevor Darrell,et al.  The pyramid match kernel: discriminative classification with sets of image features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  W. Bruce Croft,et al.  A Markov random field model for term dependencies , 2005, SIGIR '05.

[6]  W. Bruce Croft,et al.  Linear feature-based models for information retrieval , 2007, Information Retrieval.

[7]  Qiang Wu,et al.  Adapting boosting for information retrieval measures , 2010, Information Retrieval.

[8]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[9]  W. Bruce Croft,et al.  Parameterized concept weighting in verbose queries , 2011, SIGIR.

[10]  W. Bruce Croft,et al.  Effective query formulation with multiple information sources , 2012, WSDM '12.

[11]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[12]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[15]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[16]  James Allan,et al.  Entity query feature expansion using knowledge base links , 2014, SIGIR.

[17]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[18]  M. de Rijke,et al.  An Introduction to Click Models for Web Search: SIGIR 2015 Tutorial , 2015, SIGIR.

[19]  James P. Callan,et al.  Learning to Reweight Terms with Distributed Representations , 2015, SIGIR.

[20]  M. de Rijke,et al.  Click Models for Web Search , 2015, Click Models for Web Search.

[21]  Jiafeng Guo,et al.  Analysis of the Paragraph Vector Model for Information Retrieval , 2016, ICTIR.

[22]  Xueqi Cheng,et al.  A Study of MatchPyramid Models on Ad-hoc Retrieval , 2016, ArXiv.

[23]  W. Bruce Croft,et al.  Semantic Matching by Non-Linear Word Transportation for Information Retrieval , 2016, CIKM.

[24]  Bhaskar Mitra,et al.  Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR) , 2016, SIGIR Forum.

[25]  Xueqi Cheng,et al.  Text Matching as Image Recognition , 2016, AAAI.

[26]  Nick Craswell,et al.  Query Expansion with Locally-Trained Word Embeddings , 2016, ACL.

[27]  W. Bruce Croft,et al.  A Deep Relevance Matching Model for Ad-hoc Retrieval , 2016, CIKM.

[28]  Bhaskar Mitra,et al.  SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) , 2017, SIGIR.

[29]  Tie-Yan Liu,et al.  Word-Entity Duet Representations for Document Ranking , 2017, SIGIR.

[30]  W. Bruce Croft,et al.  Relevance-based Word Embedding , 2017, SIGIR.

[31]  W. Bruce Croft,et al.  Neural Ranking Models with Weak Supervision , 2017, SIGIR.

[32]  Nick Craswell,et al.  Learning to Match using Local and Distributed Representations of Text for Web Search , 2016, WWW.

[33]  Allan Hanbury,et al.  Word Embedding Causes Topic Shifting; Exploit Global Context! , 2017, SIGIR.

[34]  Zhiyuan Liu,et al.  End-to-End Neural Ad-hoc Ranking with Kernel Pooling , 2017, SIGIR.