Selection of optimal hyper-parameter values of support vector machine for sentiment analysis tasks using nature-inspired optimization methods

Sentiment analysis and classification task is used in recommender systems to analyze movie reviews, tweets, Facebook posts, online product reviews, blogs, discussion forums, and online comments in social networks. Usually, the classification is performed using supervised machine learning methods such as support vector machine (SVM) classifier, which have many distinct parameters. The selection of the values for these parameters can greatly influence the classification accuracy and can be addressed as an optimization problem. Here we analyze the use of three heuristics, nature-inspired optimization techniques, cuckoo search optimization (CSO), ant lion optimizer (ALO), and polar bear optimization (PBO), for parameter tuning of SVM models using various kernel functions. We validate our approach for the sentiment classification task of Twitter dataset. The results are compared using classification accuracy metric and the Nemenyi test.

[1]  Hsinchun Chen,et al.  Sentiment analysis in multiple languages: Feature selection for opinion classification in Web forums , 2008, TOIS.

[2]  Wojciech Czarnecki,et al.  Robust optimization of SVM hyperparameters in the classification of bioactive compounds , 2015, Journal of Cheminformatics.

[3]  Robertas Damasevicius,et al.  Sentiment Analysis of Lithuanian Texts Using Traditional and Deep Learning Approaches , 2019, Comput..

[4]  Kuan-Cheng Lin,et al.  Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Artificial Fish Swarm Algorithms , 2015 .

[5]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[6]  Francesco Archetti,et al.  Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization , 2019, Comput. Oper. Res..

[7]  Chin-Chun Chang,et al.  Tuning of the hyperparameters for L2-loss SVMs with the RBF kernel by the maximum-margin principle and the jackknife technique , 2015, Pattern Recognit..

[8]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[9]  Marcin Woźniak,et al.  Graphic object feature extraction system based on Cuckoo Search Algorithm , 2016, Expert Syst. Appl..

[10]  Ming-Yuan Cho,et al.  Feature Selection and Parameters Optimization of SVM Using Particle Swarm Optimization for Fault Classification in Power Distribution Systems , 2017, Comput. Intell. Neurosci..

[11]  Chuandong Qin,et al.  Selecting Parameters of an Improved Doubly Regularized Support Vector Machine based on Chaotic Particle Swarm Optimization Algorithm , 2017, J. Univers. Comput. Sci..

[12]  Zixue Cheng,et al.  CNN for situations understanding based on sentiment analysis of twitter data , 2017 .

[13]  Guangwen Yang,et al.  An EnKF-based scheme to optimize hyper-parameters and features for SVM classifier , 2017, Pattern Recognit..

[14]  Mizuki Morita,et al.  Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter , 2011, EMNLP.

[15]  Robertas Damaševičius,et al.  Optimization of SVM parameters for recognition of regulatory DNA sequences , 2010 .

[16]  Estevam R. Hruschka,et al.  Tweet sentiment analysis with classifier ensembles , 2014, Decis. Support Syst..

[17]  Seifedine Kadry,et al.  On the Improvement of Cyclomatic Complexity Metric , 2013 .

[18]  Songbo Tan,et al.  A survey on sentiment detection of reviews , 2009, Expert Syst. Appl..

[19]  Ali Selamat,et al.  Twitter Feature Selection and Classification Using Support Vector Machine for Aspect-Based Sentiment Analysis , 2016, IEA/AIE.

[20]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[21]  Robertas Damasevicius,et al.  Radiation heat transfer optimization by the use of modified ant lion optimizer , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[22]  Yi-Hung Huang,et al.  Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Cat Swarm Optimization , 2015, Int. J. Distributed Sens. Networks.

[23]  Albert Bifet,et al.  Sentiment Knowledge Discovery in Twitter Streaming Data , 2010, Discovery Science.

[24]  Aboul Ella Hassanien,et al.  A BA-based algorithm for parameter optimization of Support Vector Machine , 2017, Pattern Recognit. Lett..

[25]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  A hybrid meta-learning architecture for multi-objective optimization of SVM parameters , 2014, Neurocomputing.

[26]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[27]  Jin Zhang,et al.  An empirical study of sentiment analysis for chinese documents , 2008, Expert Syst. Appl..

[28]  Jacques Wainer,et al.  Empirical Evaluation of Resampling Procedures for Optimising SVM Hyperparameters , 2017, J. Mach. Learn. Res..

[29]  João Francisco Valiati,et al.  Document-level sentiment classification: An empirical comparison between SVM and ANN , 2013, Expert Syst. Appl..

[30]  Marcin Wozniak,et al.  Polar Bear Optimization Algorithm: Meta-Heuristic with Fast Population Movement and Dynamic Birth and Death Mechanism , 2017, Symmetry.

[31]  Rosa M. Carro,et al.  Sentiment analysis in Facebook and its application to e-learning , 2014, Comput. Hum. Behav..

[32]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[33]  Alfonso Rojas-Domínguez,et al.  Optimal Hyper-Parameter Tuning of SVM Classifiers With Application to Medical Diagnosis , 2018, IEEE Access.

[34]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[35]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .

[36]  Mahamad Nabab Alam,et al.  A novel differential particle swarm optimization for parameter selection of support vector machines for monitoring metal-oxide surge arrester conditions , 2018, Swarm Evol. Comput..