Automated Personalized Assessment of Computational Thinking MOOC Assignments

One of the major hurdles toward automatic semantic understanding of computer programs is the lack of knowledge about what constitutes functional equivalence of code segments. We postulate that a sound knowledgebase can be used to deductively understand code segments in a hierarchical fashion by first de-constructing a code and then reconstructing it from elementary knowledge and equivalence rules of elementary code segments. The approach can also be engineered to produce computable programs from conceptual and abstract algorithms as an inverse function. In this paper, we introduce the core idea behind the MindReader online assessment system that is able to understand a wide variety of elementary algorithms students learn in their entry level programming classes such as Java, C++ and Python. The MindReader system is able to assess student assignments and guide them how to develop correct and better code in real time without human assistance.

[1]  Mexhid Ferati,et al.  Assessing the effectiveness of distributed pair programming for an online informatics curriculum , 2010, INROADS.

[2]  Carol Hulls,et al.  Interactive online tutorial assistance for a first programming course , 2005, IEEE Transactions on Education.

[3]  Stefan Wagner,et al.  How are functionally similar code clones syntactically different? An empirical study and a benchmark , 2016, PeerJ Comput. Sci..

[4]  João Carlos Gluz,et al.  A Student Model for Teaching Natural Deduction Based on a Prover That Mimics Student Reasoning , 2014, Intelligent Tutoring Systems.

[5]  Jason Sharp Traditional, Online, and Flipped: A Preliminary Analysis of Instructional Approaches on Student Performance in a C# Programming Course , 2016, AMCIS.

[6]  Waleed Farag Comparing achievement of intended learning outcomes in online programming classes with blended offerings , 2012, SIGITE '12.

[7]  Daryl J. D'Souza,et al.  Instructor perspectives of multiple-choice questions in summative assessment for novice programmers , 2010, Comput. Sci. Educ..

[8]  Ivana Simonova Multiple-choice testing: Knowledge, or random choice? , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[9]  Torsten Görg Interprocedural PDG-based Code Clone Detection , 2016, Softwaretechnik-Trends.

[10]  Rebecca Smith,et al.  Data-Driven Test Case Generation for Automated Programming Assessment , 2016, ITiCSE.

[11]  José Manuel Azevedo e-Assessment in Mathematics Courses with Multiple-choice Questions Tests , 2015, CSEDU.

[12]  Shruti Mehta Development and Assessment of Interactive Web-based Problem Solvers in Reinforcing Math and Physics Concepts , 2001, WebNet.

[13]  Fangfang Zhang,et al.  Program Logic Based Software Plagiarism Detection , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[14]  Stefan Woltran,et al.  Complexity Results for Checking Equivalence of Stratified Logic Programs , 2007, IJCAI.

[15]  Dalila Souilem,et al.  Workflow approach to design automatic tutor in e-learning environment , 2016, 2016 International Conference on Control, Decision and Information Technologies (CoDIT).

[16]  Hardi Hungar Complexity of Proving Program Correctness , 1991, TACS.

[17]  Ana Sánchez,et al.  A comparative analysis of the consistency and difference among online self-, peer-, external- and instructor-assessments: The competitive effect , 2016, Comput. Hum. Behav..

[18]  Wlodzimierz Drabent,et al.  Proving correctness and completeness of normal programs - a declarative approach , 2005, Theory Pract. Log. Program..

[19]  Hasan M. Jamil Smart Assessment of and Tutoring for Computational Thinking MOOC Assignments using MindReader , 2017, ArXiv.

[20]  Carlos R. Rivero,et al.  Automated Personalized Feedback in Introductory Java Programming MOOCs , 2017, 2017 IEEE 33rd International Conference on Data Engineering (ICDE).

[21]  Pavol Návrat,et al.  Online programming exercises for summative assessment in university courses , 2014, CompSysTech.

[22]  Juha Sorva,et al.  Embedded questions in ebooks on programming: useful for a) summative assessment, b) formative assessment, or c) something else? , 2015, Koli Calling.

[23]  Sartaj Sahni,et al.  On the Computational Complexity of Program Scheme Equivalence , 1980, SIAM J. Comput..