Revising computer science learning objects from learner interaction data

Learning objects (LO) have previously been used to help deliver introductory computer science (CS) courses to students. Students in such introductory CS courses have diverse backgrounds and characteristics requiring revision to LO content and assessment to promote learning in all students. However, revising LOs in an ad hoc manner could make student learning harder for subsequent deployments. To address this problem, we present a systematic revision process for LOs (LOSRP) using proven techniques from educational research including Bloom's Taxonomy levels, item-total correlation, and Cronbach's Alpha. LOSRP uses these validation methods to answer seven questions in order to diagnose what needs to be revised in the LO. Then, LOSRP provides guidelines on revising LOs for each of the seven questions. As an example, we discuss how LOSRP was used to revise the content and assessment for 16 LOs deployed to over 400 students in introductory CS courses in 2009. Lastly, although initially designed for LO revision, we briefly discuss how LOSRP could be used for assessment revision in intelligent tutoring systems.

[1]  Leen-Kiat Soh,et al.  Empirical usage metadata in learning objects , 2009, 2009 39th IEEE Frontiers in Education Conference.

[2]  Leen-Kiat Soh,et al.  Design, development, and validation of a learning object for CS1 , 2005, ITiCSE '05.

[3]  Bill Z. Manaris,et al.  Bloom's taxonomy revisited: specifying assessable learning objectives in computer science , 2008, SIGCSE '08.

[4]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[5]  Ronald J. Leach,et al.  An advanced assessment tool and process , 2010, SIGCSE.

[6]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[7]  Lee Dee Miller,et al.  Intelligent Learning Object Guide (iLOG): A Framework for Automatic Empirically-Based Metadata Generation , 2009, AIED.

[8]  Croudace Tim,et al.  What is applied psychometrics , 2010 .

[9]  Mustafa Murat Inceoglu,et al.  Reusable Learning Objects (RLOs) for Computer Science Students , 2006, ICCSA.

[10]  Stephen H. Edwards,et al.  Designing an adaptive learning module to teach software testing , 2006, SIGCSE '06.

[11]  Mark Guzdial,et al.  Developing a validated assessment of fundamental CS1 concepts , 2010, SIGCSE.

[12]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[13]  Darren Mundy.  Using learning objects to support introductory computer architecture education , 2006 .

[14]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .