Ontology-Based Multiple Choice Question Generation

With recent advancements in Semantic Web technologies, a new trend in MCQ item generation has emerged through the use of ontologies. Ontologies are knowledge representation structures that formally describe entities in a domain and their relationships, thus enabling automated inference and reasoning. Ontology-based MCQ item generation is still in its infancy, but substantial research efforts are being made in the field. However, the applicability of these models for use in an educational setting has not been thoroughly evaluated. In this paper, we present an experimental evaluation of an ontology-based MCQ item generation system known as OntoQue. The evaluation was conducted using two different domain ontologies. The findings of this study show that ontology-based MCQ generation systems produce satisfactory MCQ items to a certain extent. However, the evaluation also revealed a number of shortcomings with current ontology-based MCQ item generation systems with regard to the educational significance of an automatically constructed MCQ item, the knowledge level it addresses, and its language structure. Furthermore, for the task to be successful in producing high-quality MCQ items for learning assessments, this study suggests a novel, holistic view that incorporates learning content, learning objectives, lexical knowledge, and scenarios into a single cohesive framework.

[1]  Marija Cubric,et al.  SeMCQ – Protégé Plugin for Automatic Ontology-Driven Multiple Choice Question Tests Generation , 2009 .

[2]  Eduard H. Hovy,et al.  Toward Semantics-Based Answer Pinpointing , 2001, HLT.

[3]  Andreas Papasalouros,et al.  Automatic Generation Of Multiple Choice Questions From Domain Ontologies , 2008, e-Learning.

[4]  Claudia Soria,et al.  Lexical Markup Framework (LMF) , 2006, LREC.

[5]  Lewis R. Aiken,et al.  Testing with Multiple-Choice Items. , 1987 .

[6]  Holger Knublauch,et al.  The Protégé OWL Plugin: An Open Development Environment for Semantic Web Applications , 2004, SEMWEB.

[7]  Maha M. Al-Yahya,et al.  OntoQue: A Question Generation Engine for Educational Assesment Based on Domain Ontologies , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[8]  Mark J. Gierl,et al.  Using Automated Item Generation to Promote Principled Test Design and Development , 2010 .

[9]  Paul Buitelaar,et al.  Towards Linguistically Grounded Ontologies , 2009, ESWC.

[10]  Peter Bancroft,et al.  Multiple Choice Questions Not Considered Harmful , 2005, ACE.

[11]  Martin Kuefer,et al.  Measurement And Evaluation In Teaching , 2016 .

[12]  Liana Stanescu,et al.  Question generation for learning evaluation , 2008, 2008 International Multiconference on Computer Science and Information Technology.

[13]  Irina Leshcheva,et al.  Ontology-based Assessment Technique , 2010 .

[14]  Charles D. Bailey,et al.  A Comparison of the Quality of Multiple-Choice Questions from CPA Exams and Textbook Test Banks , 1998 .

[15]  Maxine Eskénazi,et al.  Automatic Question Generation for Vocabulary Assessment , 2005, HLT.

[16]  Constantin Orasan,et al.  Automatic Question Pattern Generation for Ontology-based Question Answering , 2008, FLAIRS.

[17]  Clinton I. Chase,et al.  Developing and Using Tests Effectively: A Guide for Faculty , 1992 .

[18]  John H. Wolfe,et al.  An Aid to Independent Study through Automatic Question Generation (AUTOQUEST). , 1975 .

[19]  R. U S L A N M I T K O V,et al.  A computer-aided environment for generating multiple-choice test items , 2005 .

[20]  Slavomir Stankov,et al.  Dynamic test generation over ontology-based knowledge representation in authoring shell , 2009, Expert Syst. Appl..

[21]  Mark J. Gierl,et al.  Developing a Taxonomy of Item Model Types to Promote Assessment Engineering , 2008 .

[22]  Le An Ha,et al.  A computer-aided environment for generating multiple-choice test items , 2006, Natural Language Engineering.

[23]  Amruth N. Kumar Generation of problems, answers, grade, and feedback---case study of a fully automated tutor , 2005, JERC.