Smart Assessment of and Tutoring for Computational Thinking MOOC Assignments using MindReader

One of the major hurdles toward automatic semantic understanding of computer programs is the lack of knowledge about what constitutes functional equivalence of code segments. We postulate that a sound knowledgebase can be used to deductively understand code segments in a hierarchical fashion by first de-constructing a code and then reconstructing it from elementary knowledge and equivalence rules of elementary code segments. The approach can also be engineered to produce computable programs from conceptual and abstract algorithms as an inverse function. In this paper, we introduce the core idea behind the MindReader online assessment system that is able to understand a wide variety of elementary algorithms students learn in their entry level programming classes such as Java, C++ and Python. The MindReader system is able to assess student assignments and guide them how to develop correct and better code in real time without human assistance.

[1]  Rebecca Smith,et al.  Data-Driven Test Case Generation for Automated Programming Assessment , 2016, ITiCSE.

[2]  Carol Hulls,et al.  Interactive online tutorial assistance for a first programming course , 2005, IEEE Transactions on Education.

[3]  Ivana Simonova.  Multiple-choice testing: Knowledge, or random choice? , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[4]  Philip S. Yu,et al.  GPLAG: detection of software plagiarism by program dependence graph analysis , 2006, KDD '06.

[5]  Heejo Lee,et al.  CLORIFI: software vulnerability discovery using code clone verification , 2016, Concurr. Comput. Pract. Exp..

[6]  Stefan Woltran,et al.  Complexity Results for Checking Equivalence of Stratified Logic Programs , 2007, IJCAI.

[7]  Dalila Souilem,et al.  Workflow approach to design automatic tutor in e-learning environment , 2016, 2016 International Conference on Control, Decision and Information Technologies (CoDIT).

[8]  Hardi Hungar.  Complexity of Proving Program Correctness , 1991, TACS.

[9]  Ana Sánchez,et al.  A comparative analysis of the consistency and difference among online self-, peer-, external- and instructor-assessments: The competitive effect , 2016, Comput. Hum. Behav..

[10]  Daryl J. D'Souza,et al.  Instructor perspectives of multiple-choice questions in summative assessment for novice programmers , 2010, Comput. Sci. Educ..

[11]  Stefan Wagner,et al.  How are functionally similar code clones syntactically different? An empirical study and a benchmark , 2016, PeerJ Comput. Sci..

[12]  João Carlos Gluz,et al.  A Student Model for Teaching Natural Deduction Based on a Prover That Mimics Student Reasoning , 2014, Intelligent Tutoring Systems.

[13]  Jason Sharp.  Traditional, Online, and Flipped: A Preliminary Analysis of Instructional Approaches on Student Performance in a C# Programming Course , 2016, AMCIS.

[14]  Carlos R. Rivero,et al.  Automated Personalized Feedback in Introductory Java Programming MOOCs , 2017, 2017 IEEE 33rd International Conference on Data Engineering (ICDE).

[15]  Pavol Návrat,et al.  Online programming exercises for summative assessment in university courses , 2014, CompSysTech.

[16]  Wlodzimierz Drabent,et al.  Proving correctness and completeness of normal programs – a declarative approach , 2001, Theory and Practice of Logic Programming.

[17]  Juha Sorva,et al.  Embedded questions in ebooks on programming: useful for a) summative assessment, b) formative assessment, or c) something else? , 2015, Koli Calling.

[18]  Sartaj Sahni,et al.  On the Computational Complexity of Program Scheme Equivalence , 1980, SIAM J. Comput..

[19]  Shruti Mehta.  Development and Assessment of Interactive Web-based Problem Solvers in Reinforcing Math and Physics Concepts , 2001, WebNet.

[20]  Waleed Farag.  Comparing achievement of intended learning outcomes in online programming classes with blended offerings , 2012, SIGITE '12.

[21]  José Manuel Azevedo.  e-Assessment in Mathematics Courses with Multiple-choice Questions Tests , 2015, CSEDU.

[22]  Fangfang Zhang,et al.  Program Logic Based Software Plagiarism Detection , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[23]  Gang Li,et al.  A framework for test data generation of object-oriented programs based on complete testing chain , 2016, 2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD).

[24]  Mexhid Ferati,et al.  Assessing the effectiveness of distributed pair programming for an online informatics curriculum , 2010, INROADS.

[25]  Surajit Chaudhuri,et al.  On the complexity of equivalence between recursive and nonrecursive Datalog programs , 1994, PODS '94.

[26]  Alexander Sayapin.  Multiple choice assessments: Evaluation of quality , 2013, 2013 IEEE Global Engineering Education Conference (EDUCON).

[27]  Torsten Görg.  Interprocedural PDG-based Code Clone Detection , 2016, Softwaretechnik-Trends.