Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction

Abstract This paper proposes a novel multimodal framework for rating prediction of consumer products by fusing different data sources, namely physiological signals, global reviews obtained separately for the product and its brand. The reviews posted by global viewers are retrieved and processed using Natural Language Processing (NLP) technique to compute compound score considered as global rating. Also, electroencephalogram (EEG) signals of the participants were recorded simultaneously while watching different products on computer’s screen. From EEG, valence scores in terms of product rating are obtained using self-report towards each viewed product for acquiring local rating. A higher valence score corresponds to intrinsic attractiveness of the participant towards a product. Random forest based regression techniques is used to model EEG data to build a rating prediction framework considered as local rating. Furthermore, Artificial Bee Colony (ABC) based optimization algorithm is used to boost the overall performance of the framework by fusing global and local ratings. EEG dataset of 40 participants including 25 male and 15 female is recorded while viewing 42 different products available on e-commerce website. Experiment results are encouraging and suggest that the proposed ABC optimization approach can achieve lower Root Mean Square Error (RMSE) in rating prediction as compared to individual unimodal schemes.

[1]  J. Fernando Sánchez-Rada,et al.  Enhancing deep learning sentiment analysis with ensemble techniques in social applications , 2020 .

[2]  Ruhollah Taghizadeh-Mehrjardi,et al.  Artificial bee colony feature selection algorithm combined with machine learning algorithms to predict vertical and lateral distribution of soil organic matter in South Dakota, USA , 2017 .

[3]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[4]  Erik Cambria,et al.  Deep Learning-Based Document Modeling for Personality Detection from Text , 2017, IEEE Intelligent Systems.

[5]  Qiong Wu,et al.  A random walk algorithm for automatic construction of domain-oriented sentiment lexicon , 2011, Expert Syst. Appl..

[6]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[7]  Thomas Fischer,et al.  Deep learning with long short-term memory networks for financial market predictions , 2017, Eur. J. Oper. Res..

[8]  Junyu Dong,et al.  Prediction of Sea Surface Temperature Using Long Short-Term Memory , 2017, IEEE Geoscience and Remote Sensing Letters.

[9]  S. Hargittai.  Savitzky-Golay least-squares polynomial filters in ECG signal processing , 2005, Computers in Cardiology, 2005.

[10]  Debi Prosad Dogra,et al.  Prediction of advertisement preference by fusing EEG response and sentiment analysis , 2017, Neural Networks.

[11]  Rajeev R. Raje,et al.  Towards Selecting and Recommending Online Software Services by Evaluating External Attributes , 2016, CISRC.

[12]  Erik Cambria,et al.  Multi-attention Recurrent Network for Human Communication Comprehension , 2018, AAAI.

[13]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[14]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.

[15]  Mohammad Soleymani,et al.  Affective ranking of movie scenes using physiological signals and content analysis , 2008, MS '08.

[16]  R. Polikar,et al.  Multimodal EEG, MRI and PET data fusion for Alzheimer's disease diagnosis , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[17]  Guibing Guo,et al.  A New Recommender System for 3D E-Commerce: An EEG Based Approach , 2013 .

[18]  Athena Vakali,et al.  Sentiment analysis leveraging emotions and word embeddings , 2017 .

[19]  Victor Chang,et al.  Enhancing Text Using Emotion Detected from EEG Signals , 2018, Journal of Grid Computing.

[20]  Davide Baldo,et al.  Brain Waves Predict Success of New Fashion Products: A Practical Application for the Footwear Retailing Industry , 2015 .

[21]  Erik Cambria,et al.  A review of affective computing: From unimodal analysis to multimodal fusion , 2017, Inf. Fusion.

[22]  Minara P. Anto,et al.  Product rating using sentiment analysis , 2016, 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT).

[23]  Erik Cambria,et al.  Tensor Fusion Network for Multimodal Sentiment Analysis , 2017, EMNLP.

[24]  Raquel Cervigon.  Biomedical Applications of the Discrete Wavelet Transform , 2011 .

[25]  Rudy Prabowo,et al.  Sentiment analysis: A combined approach , 2009, J. Informetrics.

[26]  Debi Prosad Dogra,et al.  A multimodal framework for sensor based sign language recognition , 2017, Neurocomputing.

[27]  Erik Cambria,et al.  A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks , 2016, COLING.

[28]  Mike Thelwall,et al.  Sentiment Analysis Is a Big Suitcase , 2017, IEEE Intelligent Systems.

[29]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[30]  Debi Prosad Dogra,et al.  Coupled HMM-based multi-sensor data fusion for sign language recognition , 2017, Pattern Recognit. Lett..

[31]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[32]  Xavier Giró-i-Nieto,et al.  From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction , 2016, Image Vis. Comput..

[33]  Sylvie Charbonnier,et al.  EEG index for control operators' mental fatigue monitoring using interactions between brain regions , 2016, Expert Syst. Appl..

[34]  Satchidananda Dehuri,et al.  ABC optimized RBF network for classification of EEG signal for epileptic seizure identification , 2017 .

[35]  Debi Prosad Dogra,et al.  A bio-signal based framework to secure mobile devices , 2017, J. Netw. Comput. Appl..

[36]  G. Berns,et al.  A Neural Predictor of Cultural Popularity , 2010 .

[37]  Kelvin K. W. Yau,et al.  Predicting electricity energy consumption: A comparison of regression analysis, decision tree and neural networks , 2007 .

[38]  Debi Prosad Dogra,et al.  Analysis of EEG signals and its application to neuromarketing , 2017, Multimedia Tools and Applications.

[39]  Lillie Dewan,et al.  Application of discrete wavelet transform for analysis of genomic sequences of Mycobacterium tuberculosis , 2016, SpringerPlus.

[40]  Dong-Hong Ji,et al.  A topic-enhanced word embedding for Twitter sentiment classification , 2016, Inf. Sci..

[41]  Partha Pratim Roy,et al.  Age and gender classification using brain–computer interface , 2019, Neural Computing and Applications.

[42]  Swaminathan Natarajan,et al.  Sentiment analysis of Facebook data using Hadoop based open source technologies , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[43]  M. Murugappan,et al.  Wireless EEG signals based Neuromarketing system using Fast Fourier Transform (FFT) , 2014, 2014 IEEE 10th International Colloquium on Signal Processing and its Applications.

[44]  Jordan J. Louviere,et al.  Consumer neuroscience: Assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking , 2013, Expert Syst. Appl..

[45]  Min-Yuh Day,et al.  Deep learning for financial sentiment analysis on finance news providers , 2016, 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[46]  Christophe Morin,et al.  Neuromarketing: The New Science of Consumer Behavior , 2011 .

[47]  Ariel Telpaz,et al.  Using EEG to Predict Consumers’ Future Choices , 2015 .

[48]  Anil Kumar,et al.  Adaptive filtering of EEG/ERP through Bounded Range Artificial Bee Colony (BR-ABC) algorithm , 2014, Digit. Signal Process..

[49]  Lei Huang,et al.  Text Classification Research with Attention-based Recurrent Neural Networks , 2018, Int. J. Comput. Commun. Control.

[50]  Victor I. Chang,et al.  Organisational sustainability modelling - An emerging service and analytics model for evaluating Cloud Computing adoption with two case studies , 2016, Int. J. Inf. Manag..

[51]  Scott A. Huettel,et al.  Neuromarketing: Ethical Implications of its Use and Potential Misuse , 2016, Journal of Business Ethics.

[52]  Samir Avdakovic,et al.  Energy Distribution of EEG Signals: EEG Signal Wavelet-Neural Network Classifier , 2010, ArXiv.