Matching Cross Network for Learning to Rank in Personal Search

Recent neural ranking algorithms focus on learning semantic matching between query and document terms. However, practical learning to rank systems typically rely on a wide range of side information beyond query and document textual features, like location, user context, etc. It is common practice to concatenate all of these features and rely on deep models to learn a complex representation. We study how to effectively and efficiently combine textual information from queries and documents with other useful but less prominent side information for learning to rank. We conduct synthetic experiments to show that: 1) neural networks are inefficient at learning the interaction between two prominent features (e.g., query and document embedding features) in the presence of other less prominent features; 2) direct application of a state-of-art method for higher-order feature generation is also inefficient. Based on the above observations, we propose a simple but effective matching cross network (MCN) method for learning to rank with side information. MCN conducts an element-wise multiplication matching of query and document embeddings and leverages a technique called latent cross to effectively learn the interaction between matching output and all side information. The approach is easy to implement, and adds minimal parameters and latency overhead to standard neural ranking architectures. We conduct extensive experiments using two of the world’s largest personal search engines, Gmail and Google Drive search, and show that each proposed component adds meaningful gains against a strong production baseline with minimal latency overhead, thereby demonstrating the practical effectiveness and efficiency of the proposed approach.

[1]  Bartłomiej Twardowski,et al.  Modelling Contextual Information in Session-Aware Recommender Systems with Neural Networks , 2016, RecSys.

[2]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[3]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[4]  Pinar Donmez,et al.  On the local optimality of LambdaRank , 2009, SIGIR.

[5]  Mihajlo Grbovic,et al.  Search Ranking And Personalization at Airbnb , 2017, RecSys.

[6]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[7]  Marc Najork,et al.  Learning to Rank with Selection Bias in Personal Search , 2016, SIGIR.

[8]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[9]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[10]  Hamed Zamani,et al.  Situational Context for Ranking in Personal Search , 2017, WWW.

[11]  Peng Zhang,et al.  IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models , 2017, SIGIR.

[12]  Marc Najork,et al.  Learning from User Interactions in Personal Search via Attribute Parameterization , 2017, WSDM.

[13]  W. Bruce Croft,et al.  Neural Ranking Models with Weak Supervision , 2017, SIGIR.

[14]  Zhen Qin,et al.  Multi-Task Learning for Email Search Ranking with Auxiliary Query Clustering , 2018, CIKM.

[15]  Xueqi Cheng,et al.  Text Matching as Image Recognition , 2016, AAAI.

[16]  Bhaskar Mitra,et al.  Neural Ranking Models with Multiple Document Fields , 2017, WSDM.

[17]  Bhaskar Mitra,et al.  An Introduction to Neural Information Retrieval , 2018, Found. Trends Inf. Retr..

[18]  Qiang Wu,et al.  Adapting boosting for information retrieval measures , 2010, Information Retrieval.

[19]  Jimmy J. Lin,et al.  Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search , 2018, AAAI.

[20]  Jia Li,et al.  Latent Cross: Making Use of Context in Recurrent Recommender Systems , 2018, WSDM.

[21]  Ebrahim Bagheri,et al.  Document Retrieval Model Through Semantic Linking , 2017, WSDM.

[22]  Xuanhui Wang,et al.  Combining Decision Trees and Neural Networks for Learning-to-Rank in Personal Search , 2019, KDD.

[23]  Zhiyuan Liu,et al.  End-to-End Neural Ad-hoc Ranking with Kernel Pooling , 2017, SIGIR.

[24]  Deng Cai,et al.  What to Do Next: Modeling User Behaviors by Time-LSTM , 2017, IJCAI.

[25]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[26]  Hongbo Deng,et al.  Ranking Relevance in Yahoo Search , 2016, KDD.

[27]  W. Bruce Croft,et al.  Learning a Deep Listwise Context Model for Ranking Refinement , 2018, SIGIR.

[28]  Gianmaria Silvello,et al.  Probabilistic Word Embeddings in Neural IR: A Promising Model That Does Not Work as Expected (For Now) , 2019, ICTIR.

[29]  Zhiyuan Liu,et al.  Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search , 2018, WSDM.

[30]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[31]  Nick Craswell,et al.  Learning to Match using Local and Distributed Representations of Text for Web Search , 2016, WWW.

[32]  Zhen Qin,et al.  Personalized Online Spell Correction for Personal Search , 2019, WWW.

[33]  Stephen E. Robertson,et al.  Simple BM25 extension to multiple weighted fields , 2004, CIKM '04.