Lexical based automated teaching evaluation via students’ short reviews

Student evaluations of teaching (SET) have become a popular approach to assess faculties’ teaching. Question‐score‐based questionnaire is the most common SET measure adopted in universities. However, it fails to cover important facets of teaching process that not mentioned in the predefined questionnaire, which can be substantially obtained from students’ short reviews. In this paper, we propose two lexical‐based methods, specifically knowledge‐based and machine learning‐based, to automatically extract opinions from short reviews. Furthermore, the diversity of reviews’ themes and styles of same sentiment polarity reviews can be observed from the extracted opinion results. The experimental results show that the proposed methods are able to achieve accuracies of 78.13 and 84.78%, respectively in the task of student review sentiment classification. Further investigation on linguistic features shows that reviews with same sentiment polarity shares similar language patterns. Finally, we present an application scenario in real SET process by utilizing aforementioned methods and discoveries.

[1]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[2]  D. Rumelhart.  Learning internal representations by back-propagating errors , 1986 .

[3]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[4]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[5]  Kathleen R. McKeown,et al.  Predicting the semantic orientation of adjectives , 1997 .

[6]  J. C. BurgesChristopher.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[7]  J. C. Ory,et al.  Teaching Evaluation: Past, Present, and Future , 2000 .

[8]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[9]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[10]  Qiang Dong,et al.  HowNet - a hybrid language and knowledge resource , 2003, International Conference on Natural Language Processing and Knowledge Engineering, 2003. Proceedings. 2003.

[11]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[12]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[13]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[14]  Hsin-Hsi Chen,et al.  Opinion Extraction, Summarization and Tracking in News and Blog Corpora , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[15]  Hsin-Hsi Chen,et al.  Mining opinions from the Web: Beyond relevance retrieval , 2007, J. Assoc. Inf. Sci. Technol..

[16]  Jin Zhang,et al.  An empirical study of sentiment analysis for chinese documents , 2008, Expert Syst. Appl..

[17]  Haji Binali,et al.  A new significant area: Emotion detection in E-learning using opinion mining techniques , 2009, 2009 3rd IEEE International Conference on Digital Ecosystems and Technologies.

[18]  Hua Xu,et al.  Exploiting effective features for chinese sentiment classification , 2011, Expert Syst. Appl..

[19]  Yi Li,et al.  A hybrid recommendation algorithm adapted in e-learning environments , 2012, World Wide Web.

[20]  Pilar Rodríguez Marín,et al.  Extracting Emotions from Texts in E-Learning Environments , 2012, 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems.

[21]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[22]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[23]  Manasi S. Patwardhan,et al.  Multi-aspect and Multi-class Based Document Sentiment Analysis of Educational Data Catering Accreditation Process , 2013, 2013 International Conference on Cloud & Ubiquitous Computing & Emerging Technologies.

[24]  Chakrit Pong-Inwong,et al.  Teaching Senti-Lexicon for Automated Sentiment Polarity Definition in Teaching Evaluation , 2014, 2014 10th International Conference on Semantics, Knowledge and Grids.

[25]  Norsaremah Salleh,et al.  A Window into the Emergence of Agile Software Development Landscape in Indonesia , 2014, SOCO 2014.

[26]  Hui Wang,et al.  Web-Based Traffic Sentiment Analysis: Methods and Applications , 2014, IEEE Transactions on Intelligent Transportation Systems.

[27]  Longzhuang Li,et al.  Recognizing and regulating e-learners' emotions based on interactive Chinese texts in e-learning systems , 2014, Knowl. Based Syst..

[28]  Zhendong Niu,et al.  Automatic construction of domain-specific sentiment lexicon based on constrained label propagation , 2014, Knowl. Based Syst..

[29]  Tien-ming Lee,et al.  Predictors of public climate change awareness and risk perception around the world , 2015 .

[30]  Hua Xu,et al.  Chinese comments sentiment classification based on word2vec and SVMperf , 2015, Expert Syst. Appl..

[31]  Stefanie S. Boswell.  Ratemyprofessors is hogwash (but I care): Effects of Ratemyprofessors and university-administered teaching evaluations on professors , 2016, Comput. Hum. Behav..

[32]  Ron Sun,et al.  Anatomy of the Mind: a Quick Overview , 2017, Cognitive Computation.

[33]  Qingtian Zeng,et al.  A teaching evaluation method based on sentiment classification , 2016, Int. J. Comput. Sci. Math..

[34]  Zhendong Niu,et al.  A learner oriented learning recommendation approach based on mixed concept mapping and immune algorithm , 2016, Knowl. Based Syst..

[35]  Miltiadis D. Lytras,et al.  Sentiment Analysis to Evaluate Teaching Performance , 2016, Int. J. Knowl. Soc. Res..

[36]  Khin Zezawar Aung,et al.  Sentiment analysis of students' comment using lexicon based approach , 2017, 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS).

[37]  Carlos Alberto Ochoa Ortíz Zezzatti,et al.  A Sentiment Analysis Model to Analyze Students Reviews of Teacher Performance Using Support Vector Machines , 2017, DCAI.

[38]  Erik Cambria,et al.  A Review of Sentiment Analysis Research in Chinese Language , 2017, Cognitive Computation.

[39]  Hyung Seok Kim,et al.  A deep learning-based sports player evaluation model based on game statistics and news articles , 2017, Knowl. Based Syst..

[40]  Zhendong Niu,et al.  A hybrid knowledge-based recommender system for e-learning based on ontology and sequential pattern mining , 2017, Future Gener. Comput. Syst..

[41]  Yung-Hui Li,et al.  Exploring effects of discussion on visual attention, learning performance, and perceptions of students learning with STR-support , 2018, Comput. Educ..

[42]  Shueb Sheikh,et al.  Twitter sentiments related to natural calamities: Analysing tweets related to the Jammu and Kashmir floods of 2014 , 2017, Electron. Libr..

[43]  Vidhyacharan Bhaskar,et al.  Mining crisis information: A strategic approach for detection of people at risk through social media analysis , 2018 .

[44]  Aitor García Pablos,et al.  W2VLDA: Almost unsupervised system for Aspect Based Sentiment Analysis , 2017, Expert Syst. Appl..

[45]  Zhendong Niu,et al.  A hybrid recommender system for e-learning based on context awareness and sequential pattern mining , 2017, Soft Computing.