Dimension Extraction Analysis of Student Performance on Problems

We present a preliminary investigation into applying dimension extraction methods from coevolutionary algorithm theory to the analysis of student-problem performance in a computer programming instruction context. Specifically, we explore using the dimension extraction coevolutionary algorithm (DECA) from coevolution and co-optimization theory, which identifies structural relationships amongst learners and tests by constructing a geometry encoding how learner performance can be distinguished in fundamentally different ways. While DECA was developed for software learners and tests, its foundational ideas can in principle be applied to data generated by human students taking real tests. Here we apply DECA's dimension-extraction algorithm to student-problem data from four semesters of an introduction to programming course where students used an online software tutor to solve a number of predesigned problems. Dimension extraction reveals structures (dimensions) that partially align with the concepts originally designed into the problems. Preliminary results suggest the structure DECA reveals is consistent when the set of students is varied.

[1]  Cynthia Taylor,et al.  Computer science concept inventories: past and future , 2014, Comput. Sci. Educ..

[2]  Jordan B. Pollack,et al.  Emergent geometric organization and informative dimensions in coevolutionary algorithms , 2007 .

[3]  Michael C. Loui,et al.  Identifying important and difficult concepts in introductory computing courses using a delphi process , 2008, SIGCSE '08.

[4]  Matthew Hertz,et al.  Investigating factors of student learning in introductory courses , 2013, SIGCSE '13.

[5]  R. Cattell.  The three basic factor-analytic research designs-their interrelations and derivatives. , 1952, Psychological bulletin.

[6]  Gregory K. W. K. Chung,et al.  Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis , 2012, EDM 2012.

[7]  Joseph C. Kush,et al.  Structure and Diagnostic Benefits of a Normative Subtest Taxonomy Developed from the WISC-III Standardization Sample. , 1999 .

[8]  Amruth N. Kumar.  The Effectiveness of Visualization for Learning Expression Evaluation , 2015, SIGCSE.

[9]  Edwin D. de Jong,et al.  Automated Extraction of Problem Structure , 2004, GECCO.

[10]  Mark L. Davison,et al.  Multidimensional Scaling Models of Personality Responding , 1994 .

[11]  Cody Ding,et al.  Profile Analysis: Multidimensional Scaling Approach. , 2001 .

[12]  O. O. Oladipupo,et al.  Application of k Means Clustering algorithm for prediction of Students Academic Performance , 2010, ArXiv.

[13]  Jan Vahrenhold,et al.  Developing and validating test items for first-year computer science courses , 2014, Comput. Sci. Educ..

[14]  Cynthia Taylor,et al.  Leveraging open source principles for flexible concept inventory development , 2014, ITiCSE '14.

[15]  Amruth N. Kumar,et al.  Solving Code-tracing Problems and its Effect on Code-writing Skills Pertaining to Program Semantics , 2015, ITiCSE.

[16]  Mark L. Davison,et al.  Multidimensional scaling and factor models of test and item responses , 1991 .

[17]  Michael C. Loui,et al.  A psychometric evaluation of the digital logic concept inventory , 2014, Comput. Sci. Educ..

[18]  Edwin D. de Jong,et al.  Coevolutionary Principles , 2012, Handbook of Natural Computing.

[19]  R. Paul Wiegand,et al.  A Data-Driven Analysis of Informatively Hard Concepts in Introductory Programming , 2016, SIGCSE.

[20]  Edwin D. de Jong,et al.  DECA: dimension extracting coevolutionary algorithm , 2006, GECCO '06.

[21]  Edwin D. de Jong,et al.  Objective Set Compression , 2008, Multiobjective Problem Solving from Nature.