Automatic question generation and answer assessment: a survey

Learning through the internet becomes popular that facilitates learners to learn anything, anytime, anywhere from the web resources. Assessment is most important in any learning system. An assessment system can find the self-learning gaps of learners and improve the progress of learning. The manual question generation takes much time and labor. Therefore, automatic question generation from learning resources is the primary task of an automated assessment system. This paper presents a survey of automatic question generation and assessment strategies from textual and pictorial learning resources. The purpose of this survey is to summarize the state-of-the-art techniques for generating questions and evaluating their answers automatically.

[1]  Andreas Papasalouros,et al.  Automatic Generation Of Multiple Choice Questions From Domain Ontologies , 2008, e-Learning.

[2]  David Coniam A Preliminary Inquiry into Using Corpus Word Frequency Data in the Automatic Generation of English Language Cloze Tests , 2013 .

[3]  Yoshua Bengio,et al.  Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus , 2016, ACL.

[4]  Li Fei-Fei,et al.  CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Li Fei-Fei,et al.  DenseCap: Fully Convolutional Localization Networks for Dense Captioning , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Chris Dyer,et al.  The NarrativeQA Reading Comprehension Challenge , 2017, TACL.

[7]  Maxine Eskénazi,et al.  Measuring Hint Level in Open Cloze Questions , 2009, FLAIRS.

[8]  Felix Hill,et al.  Measuring abstract reasoning in neural networks , 2018, ICML.

[9]  D. Nicol E‐assessment by design: using multiple‐choice tests to good effect , 2007 .

[10]  P. K. Nizar Banu,et al.  Developing the Assessment Questions Automatically to Determine the Cognitive Level of the E-Learner Using NLP Techniques , 2020, Int. J. Serv. Sci. Manag. Eng. Technol..

[11]  Maxine Eskénazi,et al.  Automatic Generation of Cloze Question Stems , 2012, PROPOR.

[12]  Eunsol Choi,et al.  TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension , 2017, ACL.

[13]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[14]  Jason S. Chang,et al.  FAST – An Automatic Generation System for Grammar Tests , 2006, ACL.

[15]  Licheng Yu,et al.  Visual Madlibs: Fill in the Blank Description Generation and Question Answering , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  R. Gur,et al.  Development of Abbreviated Nine-Item Forms of the Raven’s Standard Progressive Matrices Test , 2012, Assessment.

[17]  Nitin Madnani,et al.  Effective Feature Integration for Automated Short Answer Scoring , 2015, NAACL.

[18]  Manisha Satish Divate,et al.  Automatic Question Generation Approaches and Evaluation Techniques , 2017 .

[19]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[20]  Shourya Roy,et al.  A Perspective on Computer Assisted Assessment Techniques for Short Free-Text Answers , 2015, CAA.

[21]  Mukta Majumder,et al.  Factual open cloze question generation for assessment of learner’s knowledge , 2017, International Journal of Educational Technology in Higher Education.

[22]  Mike Wald,et al.  Advantages and Challenges of Using e-Assessment , 2018 .

[23]  Mohd Fadzil Hassan,et al.  A survey on adaptive qualitative assessment and dynamic questions generation approaches , 2010, 2010 International Symposium on Information Technology.

[24]  Sujan Kumar Saha,et al.  Automatic Generation of Multiple Choice Questions Using Wikipedia , 2013, PReMI.

[25]  Md. Monjurul Islam,et al.  Automated essay scoring using Generalized Latent Semantic Analysis , 2010, 2010 13th International Conference on Computer and Information Technology (ICCIT).

[26]  Paul Piwek,et al.  Evaluation methodologies in Automatic Question Generation 2013-2018 , 2018, INLG.

[27]  Sachio Hirokawa,et al.  An Automated Method to Generate e-Learning Quizzes from Online Language Learner Writing , 2013, Int. J. Distance Educ. Technol..

[28]  Richard S. Zemel,et al.  Exploring Models and Data for Image Question Answering , 2015, NIPS.

[29]  Montse Maritxalar,et al.  Automatic Distractor Generation for Domain Specific Texts , 2010, IceTAL.

[30]  Tassilo Klein,et al.  Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds , 2019, ArXiv.

[31]  Manish Agarwal,et al.  Automatic Cloze-Questions Generation , 2013, RANLP.

[32]  Wei Xu,et al.  Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question , 2015, NIPS.

[33]  Siti Mariyam Shamsuddin,et al.  Deep learning-based sentiment classification of evaluative text based on Multi-feature fusion , 2019, Inf. Process. Manag..

[34]  Ganesh Ramakrishnan,et al.  Proceedings of the Workshop on Question Answering for Complex Domains , 2012 .

[35]  Sri Suning Kusumawardani,et al.  A review of an information extraction technique approach for automatic short answer grading , 2016, 2016 1st International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE).

[36]  Margaret Mitchell,et al.  Generating Natural Questions About an Image , 2016, ACL.

[37]  Eduard H. Hovy,et al.  TabMCQ: A Dataset of General Knowledge Tables and Multiple-choice Questions , 2016, ArXiv.

[38]  Alexander G. Schwing,et al.  Creativity: Generating Diverse Questions Using Variational Autoencoders , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Nelson F. Liu,et al.  Crowdsourcing Multiple Choice Science Questions , 2017, NUT@EMNLP.

[40]  Jianfeng Gao,et al.  A Human Generated MAchine Reading COmprehension Dataset , 2018 .

[41]  Bidyut Das,et al.  Automatic generation of fill‐in‐the‐blank question with corpus‐based distractors for e‐assessment to enhance learning , 2019, Comput. Appl. Eng. Educ..

[42]  Manish Agarwal,et al.  Automatic Gap-fill Question Generation from Text Books , 2011, BEA@ACL.

[43]  Guokun Lai,et al.  RACE: Large-scale ReAding Comprehension Dataset From Examinations , 2017, EMNLP.

[44]  Xiao Yang,et al.  Distractor Generation for Multiple Choice Questions Using Learning to Rank , 2018, BEA@NAACL-HLT.

[45]  Benno Stein,et al.  The Eras and Trends of Automatic Short Answer Grading , 2015, International Journal of Artificial Intelligence in Education.

[46]  Shaodi You,et al.  Automatic Generation of Grounded Visual Questions , 2016, IJCAI.

[47]  Tomoko Kojiri,et al.  Automatic Question Generation for Educational Applications - The State of Art , 2014, ICCSAMA.

[48]  Philipp Koehn,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016 .

[49]  Asmita Dhokrat,et al.  Assessment of Answers: Online Subjective Examination , 2012 .

[50]  Michael S. Bernstein,et al.  Visual7W: Grounded Question Answering in Images , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Sujan Kumar Saha,et al.  Automatic Multiple Choice Question Generation From Text: A Survey , 2020, IEEE Transactions on Learning Technologies.

[52]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[53]  Xinya Du,et al.  Learning to Ask: Neural Question Generation for Reading Comprehension , 2017, ACL.

[54]  Yoav Artzi,et al.  A Corpus for Reasoning about Natural Language Grounded in Photographs , 2018, ACL.

[55]  Md. Monjurul Islam,et al.  Automated essay scoring using Generalized Latent Semantic Analysis , 2010, CIT 2010.

[56]  Maxine Eskénazi,et al.  Automatic Question Generation for Vocabulary Assessment , 2005, HLT.

[57]  Fakhroddin Noorbehbahani,et al.  The automatic assessment of free text answers using a modified BLEU algorithm , 2011, Comput. Educ..

[58]  Mukta Majumder,et al.  A System for Generating Multiple Choice Questions: With a Novel Approach for Sentence Selection , 2015, NLP-TEA@ACL/IJCNLP.

[59]  Mukta Majumder,et al.  Automatic selection of informative sentences: The sentences that can generate multiple choice questions , 2014 .

[60]  Hiroshi Nakagawa,et al.  Assisting cloze test making with a web application , 2007 .

[61]  Issey Masuda Mora,et al.  Towards Automatic Generation of Question Answer Pairs from Images , 2016 .

[62]  Li Bin,et al.  Automated Essay Scoring Using the KNN Algorithm , 2008, 2008 International Conference on Computer Science and Software Engineering.

[63]  Martin Chodorow,et al.  C-rater: Automated Scoring of Short-Answer Questions , 2003, Comput. Humanit..

[64]  Margaret Mitchell,et al.  VQA: Visual Question Answering , 2015, International Journal of Computer Vision.

[65]  Bijan Parsia,et al.  A Systematic Review of Automatic Question Generation for Educational Purposes , 2019, International Journal of Artificial Intelligence in Education.

[66]  Jie Yang,et al.  LearningQ: A Large-Scale Dataset for Educational Question Generation , 2018, ICWSM.

[67]  Le An Ha,et al.  A computer-aided environment for generating multiple-choice test items , 2006, Natural Language Engineering.

[68]  Paul Piwek,et al.  The First Question Generation Shared Task Evaluation Challenge , 2010, Dialogue Discourse.

[69]  Jill Burstein,et al.  Handbook of Automated Essay Evaluation Current Applications and New Directions , 2018 .

[70]  Ashraf Elnagar,et al.  Deep learning for Arabic subjective sentiment analysis: Challenges and research opportunities , 2020, Appl. Soft Comput..

[71]  Erkki Sutinen,et al.  Comparison of Dimension Reduction Methods for Automated Essay Grading , 2008, J. Educ. Technol. Soc..

[72]  Ali Farhadi,et al.  IQA: Visual Question Answering in Interactive Environments , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[73]  Peter W. Foltz,et al.  Identifying Patterns For Short Answer Scoring Using Graph-based Lexico-Semantic Text Matching , 2015, BEA@NAACL-HLT.

[74]  Michael Heilman,et al.  A Selection Strategy to Improve Cloze Question Quality , 2008 .

[75]  Philip Bachman,et al.  NewsQA: A Machine Comprehension Dataset , 2016, Rep4NLP@ACL.