Jutge.org: Characteristics and Experiences

Jutge.org is an open educational online programming judge designed for students and instructors, featuring a repository of problems that is well organized by courses, topics, and difficulty. Internally, Jutge.org uses a secure and efficient architecture and integrates modern verification techniques, formal methods, static code analysis, and data mining. Jutge.org has exhaustively been used during the last decade at the Universitat Politècnica de Catalunya to strengthen the learning-by-doing approach in several courses. This paper presents the main characteristics of Jutge.org and shows its use and impact on a wide range of courses covering basic programming, data structures, algorithms, artificial intelligence, functional programming, and circuit design.

[1]  David Jackson,et al.  Grading student programs using ASSYST , 1997, SIGCSE '97.

[2]  Daniel M. Zimmerman,et al.  Toward instant gradeification , 2011, 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T).

[3]  Gerhard Zucker,et al.  Solving Digital Logic Assignments with Automatic Verification in SCORM Modules , 2009 .

[4]  Adrian Kosowski,et al.  Application of an Online Judge & Contester System in Academic Tuition , 2007, ICWL.

[5]  Stephen Dean Brown,et al.  Use of HDLs in teaching of computer hardware courses , 2003, WCAE '03.

[6]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[7]  Bernd Klauer,et al.  LogiFlash – A Flash-based Logic-Simulator for educational Purposes , 2003 .

[8]  Marco Pistore,et al.  NuSMV 2: An OpenSource Tool for Symbolic Model Checking , 2002, CAV.

[9]  W. Marsden.  I and J , 2012 .

[10]  Noraini Mohamed,et al.  The Use of Cyclomatic Complexity Metrics in Programming Performance's Assessment☆ , 2013 .

[11]  Niels Pinkwart,et al.  A Review of AI-Supported Tutoring Approaches for Learning Programming , 2013, Advanced Computational Methods for Knowledge Engineering.

[12]  M. Shepperd,et al.  A critique of cyclomatic complexity as a software metric , 1988, Softw. Eng. J..

[13]  Andrew Lim,et al.  Online Judge , 2001, Comput. Educ..

[14]  Salvador Roura,et al.  Better Feedback for Educational Online Judges , 2014, CSEDU.

[15]  Jordi Petit,et al.  A MOOC on Approaches to Machine Translation , 2015 .

[16]  Christopher Douce,et al.  Automatic test-based assessment of programming: A review , 2005, JERC.

[17]  Petri Ihantola,et al.  Review of recent systems for automatic assessment of programming assignments , 2010, Koli Calling.

[18]  Leonidas J. Guibas,et al.  Codewebs: scalable homework search for massive open online programming courses , 2014, WWW.

[19]  Jesse Fox,et al.  Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance , 2015, Comput. Educ..

[20]  David Bañeres,et al.  Experiences in Digital Circuit Design Courses: A Self-Study Platform for Learning Support , 2014, IEEE Transactions on Learning Technologies.

[21]  Jean Luca Bez,et al.  URI Online Judge Academic: A tool for algorithms and programming classes , 2014, 2014 9th International Conference on Computer Science & Education.

[22]  Cory J. Butz,et al.  A Web-Based Intelligent Tutoring System for Computer Programming , 2004, IEEE/WIC/ACM International Conference on Web Intelligence (WI'04).

[23]  Nathan Griffiths,et al.  The boss online submission and assessment system , 2005, JERC.

[24]  Enric Rodríguez-Carbonell,et al.  Fun in CS2 , 2013, CSEDU.

[25]  Carl Pixley,et al.  A theory and implementation of sequential hardware equivalence , 1992, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[26]  Dinesha Weragama,et al.  Analysing Student Programs in the PHP Intelligent Tutoring System , 2014, International Journal of Artificial Intelligence in Education.

[27]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[28]  Josep Carmona,et al.  Integrating formal verification in an online judge for e-Learning logic circuit design , 2012, SIGCSE '12.

[29]  Sumit Gulwani,et al.  Automated feedback generation for introductory programming assignments , 2013, PLDI.

[30]  Simon Thompson,et al.  Haskell: The Craft of Functional Programming , 1996 .

[31]  Johan Jeuring,et al.  Ask-Elle: an Adaptable Programming Tutor for Haskell Giving Automated Feedback , 2017, International Journal of Artificial Intelligence in Education.

[32]  Erol Özçelik,et al.  The effect of competition on learning in games , 2015, Comput. Educ..

[33]  Andrew Lim,et al.  On automated grading of programming assignments in an academic institution , 2003, Comput. Educ..