Query Rewriting via Cycle-Consistent Translation for E-Commerce Search

Nowadays e-commerce search has become an integral part of many people’s shopping routines. One critical challenge in today’s e-commerce search is the semantic matching problem where the relevant items may not contain the exact terms in the user query. In this paper, we propose a novel deep neural network based approach to query rewriting, in order to tackle this problem. Specifically, we formulate query rewriting into a cyclic machine translation problem to leverage abundant click log data. Then we introduce a novel cyclic consistent training algorithm in conjunction with state-of-the-art machine translation models to achieve the optimal performance in terms of query rewriting accuracy. In order to make it practical in industrial scenarios, we optimize the syntax tree construction to reduce computational cost and online serving latency. Offline experiments show that the proposed method is able to rewrite hard user queries into more standard queries that are more appropriate for the inverted index to retrieve. Comparing with human curated rule-based method, the proposed model significantly improves query rewriting diversity while maintaining good relevancy. Online A/B experiments show that it improves core e-commerce business metrics significantly. Since the summer of 2020, the proposed model has been launched into our search engine production, serving hundreds of millions of users.

[1]  Kang Zhang,et al.  Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning , 2020, SIGIR.

[2]  Xi Xiong,et al.  From Semantic Retrieval to Pairwise Ranking: Applying Deep Learning in E-commerce Search , 2019, SIGIR.

[3]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[4]  Ashwin K. Vijayakumar,et al.  Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models , 2016, ArXiv.

[5]  Yang Liu,et al.  Neural Machine Translation with Reconstruction , 2016, AAAI.

[6]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[7]  Jennifer Widom,et al.  SimRank: a measure of structural-context similarity , 2002, KDD.

[8]  Alejandro Pazos Sierra,et al.  Encyclopedia of Artificial Intelligence , 2008 .

[9]  Hua Ouyang,et al.  Learning to Rewrite Queries , 2016, CIKM.

[10]  P. Smith,et al.  A review of ontology based query expansion , 2007, Inf. Process. Manag..

[11]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Jiewen Wu,et al.  A Study of Ontology-based Query Expansion , 2011 .

[14]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[15]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Dat Quoc Nguyen,et al.  Search Personalization with Embeddings , 2017, ECIR.

[18]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[19]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[20]  Wei Li,et al.  Multi-Interest Network with Dynamic Routing for Recommendation at Tmall , 2019, CIKM.

[21]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[23]  Rabab Kreidieh Ward,et al.  Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[24]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[25]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[26]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[27]  Ioannis Antonellis,et al.  Simrank++: query rewriting through link analysis of the clickgraph (poster) , 2008, WWW.

[28]  Prathyusha Senthil Kumar,et al.  Query Rewriting using Automatic Synonym Extraction for E-commerce Search , 2019, eCOM@SIGIR.

[29]  Frank Nielsen,et al.  Guaranteed Bounds on the Kullback–Leibler Divergence of Univariate Mixtures , 2016, IEEE Signal Processing Letters.

[30]  Kun Gai,et al.  Learning Tree-based Deep Model for Recommender Systems , 2018, KDD.

[31]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.