论文信息 - Generating News Headlines with Recurrent Neural Networks

Generating News Headlines with Recurrent Neural Networks

We describe an application of an encoder-decoder recurrent neural network with LSTM units and attention to generating headlines from the text of news articles. We find that the model is quite effective at concisely paraphrasing news articles. Furthermore, we study how the neural network decides which input words to pay attention to, and specifically we identify the function of the different neurons in a simplified attention mechanism. Interestingly, our simplified attention mechanism performs better that the more complex attention mechanism on a held out set of articles.

Konstantin Lopyrev | Konstantin Lopyrev

[1] Guigang Zhang,et al. Deep Learning , 2016, Int. J. Semantic Comput..

[2] Geoffrey E. Hinton,et al. Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3] Fei-Fei Li,et al. Deep visual-semantic alignments for generating image descriptions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4] Jason Weston,et al. A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[5] Fei-Fei Li,et al. Deep visual-semantic alignments for generating image descriptions , 2015, CVPR.

[6] Quoc V. Le,et al. Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[7] Phil Blunsom,et al. Teaching Machines to Read and Comprehend , 2015, NIPS.

[8] Peter Kulchyski. and , 2015 .

[9] Salim Roukos,et al. Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[10] Quoc V. Le,et al. Listen, Attend and Spell , 2015, ArXiv.

[11] Christopher D. Manning,et al. Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[12] Yoshua Bengio,et al. Joint Training of Deep Boltzmann Machines , 2012, ArXiv.

[13] Wojciech Zaremba,et al. Recurrent Neural Network Regularization , 2014, ArXiv.

[14] Samy Bengio,et al. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , 2015, NIPS.