Towards practical programming exercises and automated assessment in Massive Open Online Courses

In recent years, Massive Open Online Courses (MOOCs) have become a phenomenon presenting the prospect of free high class education to everybody. They bear a tremendous potential for teaching programming to a large and diverse audience. The typical MOOC components, such as video lectures, reading material, and easily assessable quizzes, however, are not sufficient for proper programming education. To learn programming, participants need an option to work on practical programming exercises and to solve actual programming tasks. It is crucial that the participants receive proper feedback on their work in a timely manner. Without a tool for automated assessment of programming assignments, the teaching teams would be restricted to offer optional ungraded exercises only. The paper at hand sketches scenarios how practical programming exercises could be provided and examines the landscape of potentially helpful tools in this context. Automated assessment has a long record in the history of computer science education. We give an overview of existing tools in this field and also explore the question what can and/or should be assessed.

[1]  Robert C. Martin.  Agile Software Development, Principles, Patterns, and Practices , 2002 .

[2]  Julie Zelenski,et al.  The quest for excellence in designing CS1/CS2 assignments , 1996, SIGCSE '96.

[3]  Ling Wu,et al.  CEclipse: An Online IDE for Programing in the Cloud , 2011, 2011 IEEE World Congress on Services.

[4]  Tarek Hegazy,et al.  The CourseMarker CBA System: Improvements over Ceilidh , 2004, Education and Information Technologies.

[5]  Steve Benford,et al.  The Ceilidh system for the automatic grading of students on programming courses , 1995, ACM-SE 33.

[6]  Christoph Meinel,et al.  Introducing hands-on experience to a Massive Open Online Course on openHPI , 2013, Proceedings of 2013 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE).

[7]  David Jackson,et al.  Grading student programs using ASSYST , 1997, SIGCSE '97.

[8]  Janet Rountree,et al.  Learning and Teaching Programming: A Review and Discussion , 2003, Comput. Sci. Educ..

[9]  David Patterson,et al.  Software Engineering Curriculum Technology Transfer : Lessons learned from Ebooks , MOOCs , and SPOCs , 2013 .

[10]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[11]  Christoph Meinel,et al.  Handling re-grading of automatically graded assignments in MOOCs , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[12]  Timo Aho,et al.  Designing IDE as a Service , 2013 .

[13]  Christoph Meinel,et al.  Designing MOOCs for the Support of Multiple Learning Styles , 2013, EC-TEL.

[14]  Thomas Vikberg,et al.  Scaffolding students' learning using test my code , 2013, ITiCSE '13.

[15]  Lauri Malmi,et al.  Experiences in automatic assessment on mass courses and issues for designing virtual courses , 2002, ITiCSE '02.

[16]  Amit Chauhan,et al.  Massive Open Online Courses (MOOCS): Emerging Trends in Assessment and Accreditation , 2014 .

[17]  Paul Roe,et al.  Learning to program through the web , 2005, ITiCSE '05.

[18]  Rolf Schulmeister.  The Position of xMOOCs in Educational Systems , 2014 .

[19]  Andreas Polze,et al.  A Platform for Interactive Software Experiments in Massive Open Online Courses , 2014, J. Integr. Des. Process. Sci..

[20]  Rob Miller,et al.  A multiplayer online game for teaching software engineering practices , 2014, L@S.

[21]  Birgit Vogel-Heuser,et al.  Quality despite quantity — Teaching large heterogenous classes in C programming and fundamentals in computer science , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[22]  Peter Hubwieser,et al.  The gap between knowledge and ability , 2012, Koli Calling.

[23]  Wenxin Li,et al.  Educational programming systems for learning at scale , 2014, L@S.

[24]  Jack Hollingsworth,et al.  Automatic graders for programming classes , 1960, Commun. ACM.

[25]  Tom McKlin,et al.  The challenges of using a MOOC to introduce "absolute beginners" to programming on specialized hardware , 2014, L@S.

[26]  Kirsti Ala-Mutka,et al.  A study of the difficulties of novice programmers , 2005, ITiCSE '05.

[27]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[28]  Kirsti Ala-Mutka,et al.  A Survey of Automated Assessment Approaches for Programming Assignments , 2005, Comput. Sci. Educ..

[29]  Christoph Meinel,et al.  Lightweight ad hoc assessment of practical programming skills at scale , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[30]  Nathan Griffiths,et al.  The boss online submission and assessment system , 2005, JERC.

[31]  Vreda Pieterse,et al.  Automated Assessment of Programming Assignments , 2013, CSERC.

[32]  Stephen H. Edwards,et al.  Supporting student-written tests of gui programs , 2008, SIGCSE '08.

[33]  Jaakko Kurhila,et al.  Multi-faceted support for MOOC in programming , 2012, SIGITE '12.

[34]  John English.  Automated assessment of GUI programs using JEWL , 2004, ITiCSE '04.

[35]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[36]  Joseph L. Zachary,et al.  Exploiting value-added content in an online course: introducing programming concepts via HTML and JavaScript , 2003, SIGCSE.

[37]  Nikolai Tillmann,et al.  Code hunt: gamifying teaching and learning of computer science at scale , 2014, L@S.

[38]  Stephen H. Edwards,et al.  Improving student performance by evaluating how well students test their own programs , 2003, JERC.

[39]  Kirsti Ala-Mutka,et al.  Journal of Information Technology Education Supporting Students in C++ Programming Courses with Automatic Program Style Assessment Supporting Students on C++ Programming Courses , 2022 .

[40]  Rob Miller,et al.  Real-time collaborative coding in a web IDE , 2011, UIST.

[41]  Ken Schwaber,et al.  SCRUM Development Process , 1997 .

[42]  Christopher Douce,et al.  Automatic test-based assessment of programming: A review , 2005, JERC.

[43]  Petri Ihantola,et al.  Review of recent systems for automatic assessment of programming assignments , 2010, Koli Calling.

[44]  David A. Patterson,et al.  Crossing the software education chasm , 2012, Commun. ACM.

[45]  Viktor Kuncak,et al.  Software verification and graph similarity for automated evaluation of students' assignments , 2012, Inf. Softw. Technol..

[46]  John F. Canny,et al.  ACCE: automatic coding composition evaluator , 2014, L@S.