NTUA-ISLab at SemEval-2019 Task 9: Mining Suggestions in the wild

As online customer forums and product comparison sites increase their societal influence, users are actively expressing their opinions and posting their recommendations on their fellow customers online. However, systems capable of recognizing suggestions still lack in stability. Suggestion Mining, a novel and challenging field of Natural Language Processing, is increasingly gaining attention, aiming to track user advice on online forums. In this paper, a carefully designed methodology to identify customer-to-company and customer-to-customer suggestions is presented. The methodology implements a rule-based classifier using heuristic, lexical and syntactic patterns. The approach ranked at 5th and 1st position, achieving an f1-score of 0.749 and 0.858 for SemEval-2019/Suggestion Mining sub-tasks A and B, respectively. In addition, we were able to improve performance results by combining the rule-based classifier with a recurrent convolutional neural network, that exhibits an f1-score of 0.79 for subtask A.

[1]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[2]  Paul Buitelaar,et al.  A Study of Suggestions in Opinionated Texts and their Automatic Detection , 2016, *SEMEVAL.

[3]  Niranjan Pedanekar,et al.  Wishful Thinking - Finding suggestions and ’buy’ wishes from product reviews , 2010, HLT-NAACL 2010.

[4]  Alicia Martínez Flor,et al.  A theoretical review of the speech act of suggesting: towards a taxonomy for its use in FLT , 2005 .

[5]  Paul Buitelaar,et al.  Towards the Extraction of Customer-to-Customer Suggestions from Reviews , 2015, EMNLP.

[6]  Sung-Hyon Myaeng,et al.  Mining advices from weblogs , 2012, CIKM.

[7]  Benno Stein,et al.  A Review Corpus for Argumentation Analysis , 2014, CICLing.

[8]  Paul Buitelaar,et al.  SemEval-2019 Task 9: Suggestion Mining from Online Reviews and Forums , 2019, *SEMEVAL.

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[12]  Xiaojin Zhu,et al.  May All Your Wishes Come True: A Study of Wishes and How to Recognize Them , 2009, NAACL.

[13]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[14]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[15]  Caroline Brun,et al.  Suggestion Mining: Detecting Suggestions for Improvement in Users' Comments , 2013, Res. Comput. Sci..

[16]  Venky Shankararaman,et al.  Text analytics approach to extract course improvement suggestions from students’ feedback , 2018, Research and Practice in Technology Enhanced Learning.

[17]  Pushpak Bhattacharyya,et al.  Helping each Other: A Framework for Customer-to-Customer Suggestion Mining using a Semi-supervised Deep Neural Network , 2018, ArXiv.