Next generation of e-assessment: automatic generation of questions

This paper provides a review of the state-of-the-art in automatic assessment generation. The paper focuses on and further develops methods for automatic generation of assessments from ontologies. We describe a novel approach and evaluate it by comparing it to other existing approaches. In addition, we report on our experience to evaluate the generated questions using a corpus-based method to simulate a real student trying to solve the questions.

[1]  Dennis Doverspike,et al.  THREE‐ALTERNATIVE MULTIPLE CHOICE TESTS: AN ATTRACTIVE OPTION , 1994 .

[2]  B. Davis.  Tools for Teaching , 1993 .

[3]  Bijan Parsia,et al.  Mining Ontologies for Analogy Questions: A Similarity-based Approach , 2012, OWLED.

[4]  Andreas Papasalouros,et al.  Automatic Generation Of Multiple Choice Questions From Domain Ontologies , 2008, e-Learning.

[5]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[6]  M. Paxton.  A Linguistic Perspective on Multiple Choice Questioning , 2000 .

[7]  Peter D. Turney.  Measuring Semantic Similarity by Latent Relational Analysis , 2005, IJCAI.

[8]  Marija Cubric,et al.  SeMCQ – Protégé Plugin for Automatic Ontology-Driven Multiple Choice Question Tests Generation , 2009 .

[9]  Marija Cubric,et al.  Towards automatic generation of e-assessment using semantic web technologies , 2011 .

[10]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[11]  Andreas Papasalouros,et al.  Automated Transformation of SWRL Rules into Multiple-Choice Questions , 2011, FLAIRS.

[12]  Maha M. Al-Yahya,et al.  OntoQue: A Question Generation Engine for Educational Assesment Based on Domain Ontologies , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[13]  Jerard F. Kehoe,et al.  Basic Item Analysis for Multiple-Choice Tests. ERIC/AE Digest. , 1995 .

[14]  Claus Pahl,et al.  Adaptive E-learning content generation based on semantic web technology , 2005 .

[15]  Michael L. Littman,et al.  Corpus-based Learning of Analogies and Semantic Relations , 2005, Machine Learning.

[16]  Gregory K. W. K. Chung,et al.  Assessment Applications of Ontologies , 2003 .

[17]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[18]  Dean Allemang,et al.  Semantic Web for the Working Ontologist - Effective Modeling in RDFS and OWL, Second Edition , 2011 .

[19]  Dylan Wiliam,et al.  Integrating Assessment with Learning: What Will It Take to Make It Work? , 2007 .

[20]  Terrence A. Brooks,et al.  Review of: Allemang, Dean and Hendler, James. Semantic Web for the working ontologist: modeling in RDF, RDFS and OWL. Burlington, MA: Morgan Kaufmann, 2008 , 2009, Inf. Res..

[21]  M. Miller,et al.  Measurement and Assessment in Teaching , 1994 .

[22]  Slavomir Stankov,et al.  Dynamic test generation over ontology-based knowledge representation in authoring shell , 2009, Expert Syst. Appl..

[23]  Maxine Eskénazi,et al.  Automatic Question Generation for Vocabulary Assessment , 2005, HLT.

[24]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[25]  Dekang Lin,et al.  An Information-Theoretic Definition of Similarity , 1998, ICML.

[26]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[27]  Le An Ha,et al.  A computer-aided environment for generating multiple-choice test items , 2006, Natural Language Engineering.

[28]  Joseph Lowman,et al.  Mastering the Techniques of Teaching , 1984 .

[29]  A. Tversky.  Features of Similarity , 1977 .

[30]  Roy Rada,et al.  Development and application of a metric on semantic nets , 1989, IEEE Trans. Syst. Man Cybern..

[31]  Eiichiro Sumita,et al.  Measuring Non-native Speakers’ Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions , 2005 .

[32]  T. Haladyna,et al.  How Many Options is Enough for a Multiple-Choice Test Item? , 1993 .

[33]  Tony Veale.  Analogy Generation with HowNet , 2005, IJCAI.

[34]  Cédrick Fairon,et al.  A Web-Based System for Automatic Language Skill Assessment: EVALING , 1999 .

[35]  Noah A. Smith,et al.  Automatic factual question generation from text , 2011 .

[36]  R. Mitkov,et al.  Computer-Aided Generation of Multiple-Choice Tests , 2003, International Conference on Natural Language Processing and Knowledge Engineering, 2003. Proceedings. 2003.

[37]  Bradley C. Love,et al.  CAB: Connectionist Analogy Builder , 2003, Cogn. Sci..

[38]  Claus Pahl,et al.  The Generation of E-Learning Exercise Problems from Subject Ontologies , 2006 .

[39]  D. Gentner.  Structure‐Mapping: A Theoretical Framework for Analogy* , 1983 .

[40]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[41]  Patrick Henry Winston,et al.  Learning and reasoning by analogy , 1980, CACM.

[42]  H. Martyn Cundy.  What Is X , 1959 .

[43]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[44]  Hiroshi Nakagawa,et al.  A Real-Time Multiple-Choice Question Generation For Language Testing: A Preliminary Study , 2005 .

[45]  Timo Järvinen,et al.  A non-projective dependency parser , 1997, ANLP.

[46]  Diego Calvanese,et al.  The Description Logic Handbook , 2007 .

[47]  Chao-Lin Liu,et al.  Applications of Lexical Information for Algorithmically Composing Multiple-Choice Cloze Items , 2005 .