Attention Based Recurrent Neural Networks for Online Advertising

We investigate the use of recurrent neural networks (RNNs) in the context of online advertising, where we use RNNs to map both query and ads to real valued vectors. In addition, we propose an attention network that assigns scores to different word locations according to their intent importance. The vector output is computed by a weighted sum of the vectors at each word. We perform end-to-end training of both the RNN and attention network under the guidance of user click logs. We show that the attention network improves the quality of learned vector representations evaluated by AUC on a manually labeled dataset. Moreover, we show that keywords extracted according to the attention scores are easy to interpret and significantly outperform the state-of-the-art query intent extraction methods.