Aspect Detection using Word and Char Embeddings with (Bi) LSTM and CRF

We proposed a new accurate aspect extraction method that makes use of both word and character-based embeddings. We have conducted experiments of various models of aspect extraction using LSTM and BiLSTM including CRF enhancement on five different pre-trained word embeddings extended with character embeddings. The results revealed that BiLSTM outperforms regular LSTM, but also word embedding coverage in train and test sets profoundly impacted aspect detection performance. Moreover, the additional CRF layer consistently improves the results across different models and text embeddings. Summing up, we obtained state-of-the-art F-score results for SemEval Restaurants (85%) and Laptops (80%).

[1]  John G. Breslin,et al.  A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis , 2016, EMNLP.

[2]  Francisco Javier González-Castaño,et al.  GTI at SemEval-2016 Task 5: SVM and CRF for Aspect Detection and Unsupervised Aspect-Based Sentiment Analysis , 2016, *SEMEVAL.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Xin Li,et al.  Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction , 2017, EMNLP.

[5]  Tomasz Kajdanowicz,et al.  Method for Aspect-Based Sentiment Annotation Using Rhetorical Analysis , 2017, ACIIDS.

[6]  Hwee Tou Ng,et al.  An Unsupervised Neural Attention Model for Aspect Extraction , 2017, ACL.

[7]  Xiaokui Xiao,et al.  Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis , 2016, EMNLP.

[8]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[9]  Roland Vollgraf,et al.  Contextual String Embeddings for Sequence Labeling , 2018, COLING.

[10]  Ming Zhou,et al.  Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction , 2016, IJCAI.

[11]  Kiyoaki Shirai,et al.  PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis , 2015, EMNLP.

[12]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[13]  Maryna Chernyshevich,et al.  IHS R&D Belarus: Cross-domain Extraction of Product Features using Conditional Random Fields , 2014 .

[14]  Iryna Gurevych,et al.  Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields , 2010, EMNLP.

[15]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[16]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[17]  Jure Leskovec,et al.  Hidden factors and hidden topics: understanding rating dimensions with review text , 2013, RecSys.

[18]  Wang Ling,et al.  Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation , 2015, EMNLP.

[19]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[20]  Lior Rokach,et al.  Dependency Tree-Based Rules for Concept-Level Aspect-Based Sentiment Analysis , 2014, SemWebEval@ESWC.

[21]  Mitchell P. Marcus,et al.  Text Chunking using Transformation-Based Learning , 1995, VLC@ACL.

[22]  Caroline Brun,et al.  XRCE: Hybrid Classification for Aspect-based Sentiment Analysis , 2014, *SEMEVAL.

[23]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[24]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[25]  Noah A. Smith,et al.  Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs , 2015, EMNLP.

[26]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[27]  Zhiqiang Toh,et al.  DLIREC: Aspect Term Extraction and Term Polarity Classification System , 2014, *SEMEVAL.

[28]  Hai Ye,et al.  Dependency-Tree Based Convolutional Neural Networks for Aspect Term Extraction , 2017, PAKDD.

[29]  Robyn Speer,et al.  ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge , 2017, *SEMEVAL.

[30]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.