Sentiment Analysis through Transfer Learning for Turkish Language

Sentiment Analysis (SA) has received much attention in recent years. In this paper, we proposed a model based on the transfer learning technique to address SA problem. First, we utilize word embeddings that are trained on 322K documents from Turkish Wikipedia. The model employs a regular Long Short-Term Memory (LSTM) with dropout. Secondly, we fine-tuned the pre-trained language model on two different target datasets (restaurant and product reviews) independently. Finally, the LSTM is trained to classify reviews according to positive and negative sentiments and its associated performance is assessed. This study is also considered to be the important attempt that uses transfer learning by applying a fine-tuning technique and deep learning architecture to address SA problem for Turkish Language.

[1]  Rui Xia,et al.  Ensemble of feature sets and classification algorithms for sentiment classification , 2011, Inf. Sci..

[2]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[3]  Hannaneh Hajishirzi,et al.  Question Answering through Transfer Learning from Large Fine-grained Supervision Data , 2017, ACL.

[4]  Arzucan Özgür,et al.  Word Polarity Detection Using a Multilingual Approach , 2013, CICLing.

[5]  Adam Kilgarriff,et al.  of the European Chapter of the Association for Computational Linguistics , 2006 .

[6]  Sheng Li,et al.  Supervised Transfer Learning for Product Information Question Answering , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[7]  Xiaolong Wang,et al.  Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach , 2011, CIKM '11.

[8]  Richard Socher,et al.  Regularizing and Optimizing LSTM Language Models , 2017, ICLR.

[9]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[10]  Prem Melville,et al.  Sentiment analysis of blogs by combining lexical knowledge with text classification , 2009, KDD.

[11]  Bing Liu,et al.  Sentiment Analysis and Opinion Mining , 2012, Synthesis Lectures on Human Language Technologies.

[12]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[13]  Harith Alani,et al.  Semantic Sentiment Analysis of Twitter , 2012, SEMWEB.

[14]  Ausif Mahmood,et al.  Deep Learning approach for sentiment analysis of short texts , 2017, 2017 3rd International Conference on Control, Automation and Robotics (ICCAR).

[15]  Cumali Türkmenoğlu,et al.  Sentiment Analysis in Turkish Media , 2014 .

[16]  Rudy Prabowo,et al.  Sentiment analysis: A combined approach , 2009, J. Informetrics.

[17]  Paul Rayson,et al.  Lancaster A at SemEval-2017 Task 5: Evaluation metrics matter: predicting sentiment from financial news headlines , 2017, *SEMEVAL.

[18]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[19]  Alessandro Moschitti,et al.  Twitter Sentiment Analysis with Deep Convolutional Neural Networks , 2015, SIGIR.

[20]  Ismail Hakki Toroslu,et al.  Sentiment Analysis of Turkish Political News , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[21]  Abdessalam Bouchekif,et al.  Epita at SemEval-2018 Task 1: Sentiment Analysis Using Transfer Learning Approach , 2018, *SEMEVAL.

[22]  K. Robert Lai,et al.  Refining Word Embeddings for Sentiment Analysis , 2017, EMNLP.

[23]  J. Fernando Sánchez-Rada,et al.  Enhancing deep learning sentiment analysis with ensemble techniques in social applications , 2020 .

[24]  Björn W. Schuller,et al.  Knowledge-Based Approaches to Concept-Level Sentiment Analysis , 2013, IEEE Intell. Syst..

[25]  Claudiu Musat,et al.  Goal-Oriented Chatbot Dialog Management Bootstrapping with Transfer Learning , 2018, IJCAI.

[26]  Mahmut Cetin,et al.  Active learning for Turkish sentiment analysis , 2013, 2013 IEEE INISTA.

[27]  Ismail Hakki Toroslu,et al.  Transfer Learning Using Twitter Data for Improving Sentiment Classification of Turkish Political News , 2013, ISCIS.

[28]  Long Jiang,et al.  User-level sentiment analysis incorporating social networks , 2011, KDD.

[29]  Kemal Oflazer,et al.  SentiTurkNet: a Turkish polarity lexicon for sentiment analysis , 2016, Lang. Resour. Evaluation.

[30]  Alberto Del Bimbo,et al.  A multimodal feature learning approach for sentiment analysis of social network multimedia , 2016, Multimedia Tools and Applications.

[31]  Banu Diri,et al.  Sentiment analysis on Twitter , 2014, 2014 22nd Signal Processing and Communications Applications Conference (SIU).

[32]  Aurélien Lucchi,et al.  SwissCheese at SemEval-2016 Task 4: Sentiment Classification Using an Ensemble of Convolutional Neural Networks with Distant Supervision , 2016, *SEMEVAL.

[33]  Vaibhavi N Patodkar,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2016 .

[34]  Chen Yuda,et al.  Research on Chinese Micro-Blog Sentiment Analysis Based on Deep Learning , 2015, 2015 8th International Symposium on Computational Intelligence and Design (ISCID).

[35]  Lei Zhang,et al.  Combining lexicon-based and learning-based methods for twitter sentiment analysis , 2011 .

[36]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[37]  Berkant Barla Cambazoglu,et al.  A Framework for Sentiment Analysis in Turkish: Application to Polarity Detection of Movie Reviews in Turkish , 2012, ISCIS.

[38]  Abdullah Uslu,et al.  SENTIMENT ANALYSIS IN TURKISH FILM COMMENTS , 2019, 2019 27th Signal Processing and Communications Applications Conference (SIU).

[39]  Bo Xu,et al.  Recursive Deep Learning for Sentiment Analysis over Social Data , 2014, 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[40]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .

[41]  Jin Zhang,et al.  An empirical study of sentiment analysis for chinese documents , 2008, Expert Syst. Appl..

[42]  Ming Zhou,et al.  Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification , 2014, ACL.

[43]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[44]  Nikos Pelekis,et al.  DataStories at SemEval-2017 Task 6: Siamese LSTM with Attention for Humorous Text Comparison , 2017, SemEval@ACL.

[45]  Murat Saraclar,et al.  Turkish tweet sentiment analysis with word embedding and machine learning , 2017, 2017 25th Signal Processing and Communications Applications Conference (SIU).

[46]  M. Ali Akcayol,et al.  A comprehensive survey for sentiment analysis tasks using machine learning techniques , 2016, 2016 International Symposium on INnovations in Intelligent SysTems and Applications (INISTA).

[47]  Björn W. Schuller,et al.  Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis , 2017, EACL.

[48]  Onder Coban,et al.  Sentiment analysis for Turkish Twitter feeds , 2015, 2015 23nd Signal Processing and Communications Applications Conference (SIU).

[49]  Patricio Martínez-Barco,et al.  Sentiment Analysis of Spanish Tweets Using a Ranking Algorithm and Skipgrams , 2013 .

[50]  Wei Chu,et al.  Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce , 2017, WSDM.

[51]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[52]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[53]  Xin Wang,et al.  Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory , 2015, ACL.

[54]  James R. Glass,et al.  Supervised and Unsupervised Transfer Learning for Question Answering , 2017, NAACL.

[55]  Minh Vo,et al.  Combining Lexicon-Based and Learning-Based Methods for Sentiment Analysis for Product Reviews in Vietnamese Language , 2017 .

[56]  Jiebo Luo,et al.  Joint Visual-Textual Sentiment Analysis with Deep Neural Networks , 2015, ACM Multimedia.

[57]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[58]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[59]  Mustafa Sert,et al.  Sentiment analysis on microblog data based on word embedding and fusion techniques , 2017, 2017 25th Signal Processing and Communications Applications Conference (SIU).