EmbedInsight: Automated Grading of Embedded Systems Assignments

Grading in embedded systems courses typically requires a face-to-face appointment between the student and the instructor because of experimental setups that are only available in laboratory facilities. Such a manual grading process is an impediment to both students and instructors. Students have to wait for several days to get feedback, and instructors may spend valuable time evaluating trivial aspects of the assignment. As seen with software courses, an automated grading system can significantly improve the insights available to the instructor and encourage students to learn quickly with iterative testing. We have designed and implemented EmbedInsight, an automated grading system for embedded system courses that accommodates a wide variety of experimental setups and is scalable to MOOC-style courses. EmbedInsight employs a modular web services design that separates the user interface and the experimental setup that evaluates student assignments. We deployed and evaluated EmbedInsight for our university embedded systems course. We show that our system scales well to a large number of submissions, and students are satisfied with their overall experience.

[1]  Wang Yi,et al.  Uppaal in a nutshell , 1997, International Journal on Software Tools for Technology Transfer.

[2]  Chun-Yen Chang,et al.  Assessing Creative Problem-solving with Automated Text Grading , 2008 .

[3]  Sanjoy K. Baruah,et al.  The Non-preemptive Scheduling of Periodic Tasks upon Multiprocessors , 2006, Real-Time Systems.

[4]  E. D. Willink,et al.  The waveform description language: moving from implementation to specification , 2001, 2001 MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No.01CH37277).

[5]  Leah S. Larkey,et al.  Automatic essay grading using text categorization techniques , 1998, SIGIR '98.

[6]  Stephen H. Edwards,et al.  Web-CAT: automatically grading programming assignments , 2008, SIGCSE 2008.

[7]  Roy Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures"; Doctoral dissertation , 2000 .

[8]  Motaz Saad,et al.  Performance Evaluation of RESTful Web Services for Mobile Devices , 2010, Int. Arab. J. e Technol..

[9]  A. Engel,et al.  Pulse Width Modulation , 2000 .

[10]  V. Shute.  Focus on Formative Feedback , 2008 .

[11]  Stephen H. Edwards,et al.  Web-CAT: automatically grading programming assignments , 2008, ITiCSE.

[12]  Matthew H. Netkow,et al.  Xest: an automated framework for regression testing of embedded software , 2010, WESE '10.

[13]  Adam Dunkels,et al.  Contiki - a lightweight and flexible operating system for tiny networked sensors , 2004, 29th Annual IEEE International Conference on Local Computer Networks.

[14]  Jian Sun.  Pulse-Width Modulation , 2012 .

[15]  Jonathan W. Valvano.  Embedded Systems - Shape The World , 2014 .

[16]  Sumit Gulwani,et al.  Automated feedback generation for introductory programming assignments , 2012, PLDI.

[17]  Kirsti Ala-Mutka,et al.  A Survey of Automated Assessment Approaches for Programming Assignments , 2005, Comput. Sci. Educ..

[18]  Jack Hollingsworth,et al.  Automatic graders for programming classes , 1960, Commun. ACM.

[19]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[20]  Johan Jeuring,et al.  Towards a Systematic Review of Automated Feedback Generation for Programming Exercises , 2016, ITiCSE.

[21]  Richard G. Baraniuk,et al.  Mathematical Language Processing: Automatic Grading and Feedback for Open Response Mathematical Questions , 2015, L@S.

[22]  Jonathan W. Valvano,et al.  Teaching Embedded Systems in a MOOC Format , 2016 .

[23]  Adam Dunkels,et al.  Cross-Level Sensor Network Simulation with COOJA , 2006, Proceedings. 2006 31st IEEE Conference on Local Computer Networks.

[24]  V. Shute.  Focus on Formative Feedback , 2007 .

[25]  Chris Sangwin.  Assessing elementary algebra with STACK , 2007 .

[26]  ChengXiang Zhai,et al.  An Exploration of Automated Grading of Complex Assignments , 2016, L@S.

[27]  Petri Mähönen,et al.  Providing network connectivity for small appliances: a functionally minimized embedded Web server , 2001, IEEE Commun. Mag..

[28]  J.L. Hardison,et al.  Deploying interactive remote labs using the iLab Shared Architecture , 2008, 2008 38th Annual Frontiers in Education Conference.

[29]  Vijayalakshmi Atluri,et al.  Role-based Access Control , 1992 .

[30]  Bettina Weiss,et al.  A system for automatic testing of embedded software in undergraduate study exercises , 2005, SIGBED.

[31]  Garvit Juniwal,et al.  CPSGrader: Synthesizing temporal logic testers for auto-grading an embedded systems laboratory , 2014, 2014 International Conference on Embedded Software (EMSOFT).

[32]  Chris Wilcox.  Testing Strategies for the Automated Grading of Student Programs , 2016, SIGCSE.

[33]  Yoon Kim,et al.  Automatic pronunciation scoring for language instruction , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[34]  Lui Sha,et al.  Priority Inheritance Protocols: An Approach to Real-Time Synchronization , 1990, IEEE Trans. Computers.

[35]  Chris Wilcox,et al.  The Role of Automation in Undergraduate Computer Science Education , 2015, SIGCSE.

[36]  Stephen Travis Pope,et al.  A Description of the Model-View-Controller User Interface Paradigm in the Smalltalk-80 System , 1998 .