Extracting Important Tweets for News Writers using Recurrent Neural Network with Attention Mechanism and Multi-task Learning

Social media is an important source for news writers. However, extracting useful information for news writers from the vast amount of social media information is laborious. Therefore, services that enable news writers to extract important information are desired. In this paper, we describe a method to extract tweets that include useful information for news writers. Our method uses a Recurrent Neural Network (RNN) with an attention mechanism and multi-task learning that processes each character in the tweet to estimate whether the tweet includes important information. In our experiment, we compared two types of attention mechanism and compared their types with/without multi-task learning. By our proposed method, we obtained an F-measure of 0.627, which is 0.037 higher than that of baseline method.

[1]  Wang Ling,et al.  Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation , 2015, EMNLP.

[2]  Naoaki Okazaki,et al.  Who caught a cold ? - Identifying the subject of a symptom , 2015, ACL.

[3]  Di Jiang,et al.  Dynamic multi-faceted topic discovery in twitter , 2013, CIKM.

[4]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[5]  Kwan Hui Lim,et al.  Geolocation Prediction in Twitter Using Location Indicative Words and Textual Features , 2016, NUT@COLING.

[6]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[7]  Mark Dredze,et al.  Geolocation for Twitter: Timing Matters , 2016, NAACL.

[8]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[9]  Soroush Vosoughi,et al.  Tweet2Vec: Learning Tweet Embeddings Using Character-level CNN-LSTM Encoder-Decoder , 2016, SIGIR.

[10]  Zhong Zhou,et al.  Tweet2Vec: Character-Based Distributed Representations for Social Media , 2016, ACL.

[11]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[12]  Lyndon J. B. Nixon,et al.  Character-based Neural Embeddings for Tweet Clustering , 2017, SocialNLP@EACL.

[13]  Ken-ichi Kawarabayashi,et al.  Real-Time Top-R Topic Detection on Twitter with Topic Hijack Filtering , 2015, KDD.

[14]  Junta Mizuno,et al.  WISDOM X, DISAANA and D-SUMM: Large-scale NLP Systems for Analyzing Textual Big Data , 2016, COLING.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Quoc V. Le,et al.  Multi-task Sequence to Sequence Learning , 2015, ICLR.

[17]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[18]  Kenta Oono,et al.  Chainer : a Next-Generation Open Source Framework for Deep Learning , 2015 .