Hierarchical neural query suggestion with an attention mechanism

Abstract Query suggestions help users of a search engine to refine their queries. Previous work on query suggestion has mainly focused on incorporating directly observable features such as query co-occurrence and semantic similarity. The structure of such features is often set manually, as a result of which hidden dependencies between queries and users may be ignored. We propose an Attention-based Hierarchical Neural Query Suggestion (AHNQS) model that uses an attention mechanism to automatically capture user preferences. AHNQS combines a session-level neural network and a user-level neural network into a hierarchical structure to model the short- and long-term search history of a user. We quantify the improvements of AHNQS over state-of-the-art recurrent neural network-based query suggestion baselines on the AOL query log dataset, with improvements of up to 9.66% and 12.51% in terms of Recall@10 and MRR@10, respectively; improvements are especially obvious for short sessions and inactive users with few search sessions.

[1]  M. de Rijke,et al.  A Click Sequence Model for Web Search , 2018, SIGIR.

[2]  Md. Mustafizur Rahman,et al.  Neural information retrieval: at the end of the early years , 2017, Information Retrieval Journal.

[3]  Yelong Shen,et al.  A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval , 2014, CIKM.

[4]  M. de Rijke,et al.  Mapping queries to the Linking Open Data cloud: A case study using DBpedia , 2011, J. Web Semant..

[5]  M. de Rijke,et al.  Diversifying Query Auto-Completion , 2016, ACM Trans. Inf. Syst..

[6]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[7]  Umut Ozertem,et al.  Learning to suggest: a machine learning framework for ranking query suggestions , 2012, SIGIR '12.

[8]  Aixin Sun,et al.  Enhancing Topic Modeling for Short Texts with Auxiliary Word Embeddings , 2017, ACM Trans. Inf. Syst..

[9]  Kenneth Ward Church,et al.  Query suggestion using hitting time , 2008, CIKM '08.

[10]  Xueqi Cheng,et al.  Intent-aware query similarity , 2011, CIKM '11.

[11]  Maarten de Rijke,et al.  Personalized Query Suggestion Diversification , 2017, SIGIR.

[12]  Craig MacDonald,et al.  Learning to rank query suggestions for adhoc and diversity search , 2012, Information Retrieval.

[13]  Xiangnan He,et al.  NAIS: Neural Attentive Item Similarity Model for Recommendation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[14]  M. de Rijke,et al.  Selectively Personalizing Query Auto-Completion , 2016, SIGIR.

[15]  Pu-Jen Cheng,et al.  Learning user reformulation behavior for query auto-completion , 2014, SIGIR.

[16]  Zhiyong Lu,et al.  Personalized neural language models for real-world query auto completion , 2018, NAACL.

[17]  Qi He,et al.  Web Query Recommendation via Sequential Query Prediction , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[18]  Hongyan Liu,et al.  Multi-view random walk framework for search task discovery from click-through log , 2011, CIKM '11.

[19]  Djoerd Hiemstra,et al.  Query recommendation in the information domain of children , 2014, J. Assoc. Inf. Sci. Technol..

[20]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[21]  M. de Rijke,et al.  Attention-based Hierarchical Neural Query Suggestion , 2018, SIGIR.

[22]  Dae Hoon Park,et al.  A Neural Language Model for Query Auto-Completion , 2017, SIGIR.

[23]  David M. Blei,et al.  Scalable Recommendation with Hierarchical Poisson Factorization , 2015, UAI.

[24]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[25]  Yen-Jen Oyang,et al.  Relevant term suggestion in interactive web search based on contextual information in query session logs , 2003, J. Assoc. Inf. Sci. Technol..

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Masoud Rahgozar,et al.  A query term re-weighting approach using document similarity , 2016, Inf. Process. Manag..

[28]  M. de Rijke,et al.  A Survey of Query Auto Completion in Information Retrieval , 2016, Found. Trends Inf. Retr..

[29]  Jian Xing,et al.  Effective Document Labeling with Very Few Seed Words: A Topic Model Approach , 2016, CIKM.

[30]  Bhaskar Mitra,et al.  Query Auto-Completion for Rare Prefixes , 2015, CIKM.

[31]  Enhong Chen,et al.  Context-aware query suggestion by mining click-through and session data , 2008, KDD.

[32]  Ling Liu,et al.  Query-URL Bipartite Based Approach to Personalized Query Recommendation , 2008, AAAI.

[33]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[34]  Jakob Grue Simonsen,et al.  A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion , 2015, CIKM.

[35]  Rifat Ozcan,et al.  New query suggestion framework and algorithms: A case study for an educational search engine , 2016, Inf. Process. Manag..

[36]  M. de Rijke,et al.  A Neural Click Model for Web Search , 2016, WWW.

[37]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[38]  M. de Rijke,et al.  Learning from homologous queries and semantically related terms for query auto completion , 2016, Inf. Process. Manag..

[39]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[40]  Michael R. Lyu,et al.  Learning latent semantic relations from clickthrough data for query suggestion , 2008, CIKM '08.

[41]  John Riedl,et al.  Learning preferences of new users in recommender systems: an information theoretic approach , 2008, SKDD.

[42]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[43]  Jacek Gwizdka,et al.  The use of query auto-completion over the course of search sessions with multifaceted information needs , 2017, Inf. Process. Manag..