Superlative model using word cloud for short answers evaluation in eLearning

Teachers are thrown open to abundance of free text answers which are very daunting to read and evaluate. Automatic assessments of open ended answers have been attempted in the past but none guarantees 100 % accuracy. In order to deal with the overload involved in this manual evaluation, a new tool becomes necessary. The unique superlative model discussed in this paper aims at providing improved accuracy by constructing word clouds. The model uses appropriate semantics with a visual appeal to partially automate free text evaluation. The model was applied at a K-12 school setup where the average human agreement rate was found to be 98 % and the accuracy score deviation from the mean was 2.82. This tool can be cast-off at any level starting from K-12 to higher education to evolve the way we view and evaluate answers.

[1]  Laura M. Ahearn.  Commentary: Keywords as a literacy practice in the history of anthropological theory , 2013 .

[2]  Benno Stein,et al.  The Eras and Trends of Automatic Short Answer Grading , 2015, International Journal of Artificial Intelligence in Education.

[3]  Sally E. Jordan,et al.  A comparison of human and computer marking of short free-text student responses , 2010, Comput. Educ..

[4]  BlinFrançoise,et al.  Why hasn't technology disrupted academics' teaching practices? Understanding resistance to change through the lens of activity theory , 2008 .

[5]  James R. Curran,et al.  Web Text Corpus for Natural Language Processing , 2006, EACL.

[6]  J. Piaget,et al.  The Kolb Learning Style Inventory — Version 3 . 1 2005 Technical Specifi cations , 2005 .

[7]  Walt Detmar Meurers.  Diagnosing Meaning Errors in Short Answers to Reading Comprehension Questions , 2008 .

[8]  Jennifer Nickell.  Word Clouds in Math Classrooms , 2012 .

[9]  Djoerd Hiemstra,et al.  How Different Are Language Models andWord Clouds? , 2010, ECIR.

[10]  Claudia Leacock,et al.  The joint student response analysis and recognizing textual entailment challenge: making sense of student responses in educational applications , 2016, Lang. Resour. Evaluation.

[11]  Kostas Karpouzis,et al.  Emerging Artificial Intelligence Applications in Computer Engineering - Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies , 2007, Emerging Artificial Intelligence Applications in Computer Engineering.

[12]  Tom Mitchell,et al.  Towards robust computerised marking of free-text responses , 2002 .

[13]  John T. Stasko,et al.  Guest Editors' Introduction: Discovering the Unexpected , 2007, IEEE Computer Graphics and Applications.

[14]  Timothy Baldwin,et al.  Multiword Expressions: A Pain in the Neck for NLP , 2002, CICLing.

[15]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[16]  Anne Kao,et al.  Natural Language Processing and Text Mining , 2006 .

[17]  Rada Mihalcea,et al.  Text-to-Text Semantic Similarity for Automatic Short Answer Grading , 2009, EACL.

[18]  J. W. Bakal,et al.  Use of Fuzzy Logic in Evaluating Students’ Learning Achievement , 2012 .

[19]  Linda S. Behar-Horenstein,et al.  Teaching Critical Thinking Skills In Higher Education: A Review Of The Literature , 2011 .

[20]  Kurt Hornik,et al.  Introduction to arules – A computational environment for mining association rules and frequent item sets , 2009 .

[21]  Carlo Strapparava,et al.  Automatic Assessment of Students' Free-Text Answers Underpinned by the Combination of a BLEU-Inspired Algorithm and Latent Semantic Analysis , 2005, FLAIRS Conference.

[22]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[23]  Andrew Ramsden,et al.  Using Word Clouds in Teaching and Learning , 2008 .

[24]  Willie Miller,et al.  We’ve Wordled, have you? Digital images in the library classroom , 2011 .

[25]  Concetta A. DePaolo,et al.  Get Your Head into the Clouds: Using Word Clouds for Analyzing Qualitative Assessment Data , 2014, TechTrends.

[26]  V. Komis,et al.  Examining teachers’ beliefs about ICT in education: implications of a teacher preparation programme , 2007 .

[27]  Stephen Pulman,et al.  Automarking: using computational linguistics to score short‚ free−text responses , 2003 .

[28]  Rajeev Motwani,et al.  Dynamic itemset counting and implication rules for market basket data , 1997, SIGMOD '97.

[29]  Raheel Siddiqi,et al.  Improving Learning and Teaching through Automated Short-Answer Marking , 2010 .

[30]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[31]  Randy M. Kaplan,et al.  Using Lexical Semantic Techniques to Classify Free-Responses , 1999 .

[32]  David M. Williamson,et al.  A Framework for Evaluation and Use of Automated Scoring , 2012 .

[33]  Jeannette M. Wing.  Computational thinking and thinking about computing , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[34]  Kurt Hornik,et al.  Text Mining Infrastructure in R , 2008 .

[35]  Walt Detmar Meurers,et al.  Evaluating the Meaning of Answers to Reading Comprehension Questions: A Semantics-Based Approach , 2012, BEA@NAACL-HLT.

[36]  Martin Chodorow,et al.  C-rater: Automated Scoring of Short-Answer Questions , 2003, Comput. Humanit..

[37]  Françoise Blin,et al.  Why hasn't technology disrupted academics' teaching practices Understanding resistance to change through the lens of activity theory , 2008, Comput. Educ..

[38]  C. Gütl.  e-Examiner: Towards a Fully-Automatic Knowledge Assessment Tool applicable in Adaptive E-Learning Systems , 2007 .

[39]  Carmel McNaught,et al.  Using Wordle as a Supplementary Research Tool , 2010 .

[40]  J. Kragten,et al.  Tutorial review. Calculating standard deviations and confidence intervals with a universally applicable spreadsheet technique , 1994 .

[41]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[42]  Stephen G. Pulman,et al.  Automatic Short Answer Marking , 2005, ACL 2005.

[43]  Susan Pennestri,et al.  Using Wordles to Teach Foreign Language Writing , 2011 .

[44]  D UllmanJeffrey,et al.  Dynamic itemset counting and implication rules for market basket data , 1997 .

[45]  Andrew Read,et al.  Using word clouds to develop proactive learners , 2011 .

[46]  Lisa Tsui.  Fostering Critical Thinking Through Effective Pedagogy: Evidence from Four Institutional Case Studies , 2002 .

[47]  Lisa Perry.  Using Word Clouds to Teach About Speaking Style , 2012 .

[48]  Aimee deNoyelles,et al.  Using Word Clouds in Online Discussions to Support Critical Thinking and Engagement , 2015 .

[49]  Rosheena Siddiqi,et al.  Improving Teaching and Learning through Automated Short-Answer Marking , 2010, IEEE Transactions on Learning Technologies.