Suggestion Miner at SemEval-2019 Task 9: Suggestion Detection in Online Forum using Word Graph

This paper describes the suggestion miner system that participates in SemEval 2019 Task 9 - SubTask A - Suggestion Mining from Online Reviews and Forums. The system participated in the subtasks A. This paper discusses the results of our system in the development, evaluation and post evaluation. Each class in the dataset is represented as directed unweighted graphs. Then, the comparison is carried out with each class graph which results in a vector. This vector is used as features by a machine learning algorithm. The model is evaluated on hold on strategy. The organizers randomly split (8500 instances) training set (provided to the participant in training their system) and testing set (833 instances). The test set is reserved to evaluate the performance of participants systems. During the evaluation, our system ranked 31 in the Coda Lab result of the subtask A (binary class problem). The binary class system achieves evaluation value 0.34, precision 0.87, recall 0.73 and F measure 0.78.

[1]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[2]  Christopher C. Cummins,et al.  Synthesizing benchmarks for predictive modeling , 2017, 2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[3]  George A. Vouros,et al.  Summarization system evaluation revisited: N-gram graphs , 2008, TSLP.

[4]  Paul Buitelaar,et al.  SemEval-2019 Task 9: Suggestion Mining from Online Reviews and Forums , 2019, *SEMEVAL.

[5]  Randal S. Olson,et al.  Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science , 2016, GECCO.

[6]  Paul Buitelaar,et al.  Towards the Extraction of Customer-to-Customer Suggestions from Reviews , 2015, EMNLP.

[7]  Paul Buitelaar,et al.  A Study of Suggestions in Opinionated Texts and their Automatic Detection , 2016, *SEMEVAL.

[8]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[9]  Muhammad Aleem,et al.  Graph Centrality Based Spam SMS Detection , 2019, 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST).

[10]  Valentin Jijkoun,et al.  Mining User Experiences from Online Forums: An Exploration , 2010, HLT-NAACL 2010.

[11]  George Papadakis,et al.  Content vs. context for sentiment analysis: a comparative analysis over microblogs , 2012, HT '12.

[12]  Muhammad Tanvir Afzal,et al.  Exploiting Polarity Features for Developing Sentiment Analysis Tool , 2017, EMSASW@ESWC.

[13]  Paul Buitelaar,et al.  Open Domain Suggestion Mining: Problem Definition and Datasets , 2018, ArXiv.

[14]  Paul Buitelaar,et al.  Inducing Distant Supervision in Suggestion Mining through Part-of-Speech Embeddings , 2017, ArXiv.

[15]  Muhammad Arshad Islam,et al.  Irony Detector at SemEval-2018 Task 3: Irony Detection in English Tweets using Word Graph , 2018, *SEMEVAL.