Data-Driven Test Case Generation for Automated Programming Assessment

Building high-quality test cases for programming problems is an important part of any well-built Automated Programming Assessment System. Traditionally, test cases are created by human experts or using machine auto-generation methods based on the problem definition and sample solutions. Unfortunately, the human approach can not anticipate the numerous ways that programmers can construct erroneous solutions. The machine auto-generation methods are complex, problem-specific, and time-consuming. This paper proposes a fast, simple method for generating high-quality test sets for a programming problem from an existing collection of student solutions for that problem. This paper demonstrates the effectiveness of the proposed method in online programming course assessments. The experiments showed that, when applied to large collections of such programs, the method produces concise, human-understandable test sets that provide better coverage than test sets built by experts with rich teaching experience.

[1]  G. Gopal,et al.  Automatic Test Case Generation Using Genetic Algorithm , 2013 .

[2]  Bo Liu,et al.  Using back-propagation neural networks for functional software testing , 2008, 2008 2nd International Conference on Anti-counterfeiting, Security and Identification.

[3]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[4]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[5]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[6]  Radziah Mohamad,et al.  Systematic Mapping Study in Automatic Test Case Generation , 2014, SoMeT.

[7]  Zili Zhang,et al.  A Fuzzy Logic Based Approach for Software Testing , 2007, Int. J. Pattern Recognit. Artif. Intell..

[8]  Christian Schallhart,et al.  Tools for Test Case Generation , 2004, Model-Based Testing of Reactive Systems.

[9]  Aniket Kittur,et al.  Crowdsourcing, collaboration and creativity , 2010, XRDS.

[10]  Kartheek Muthyala.  A NOVEL APPROACH TO TEST SUITE REDUCTION USING DATA MINING , 2011 .

[11]  Scott Rixner,et al.  An environment for learning interactive programming , 2014, SIGCSE '14.

[12]  Andreas Zeller,et al.  Mutation-Driven Generation of Unit Tests and Oracles , 2010, IEEE Transactions on Software Engineering.

[13]  Lionel C. Briand,et al.  Random Testing: Theoretical Results and Practical Implications , 2012, IEEE Transactions on Software Engineering.

[14]  Varun Aggarwal,et al.  A system to grade computer programming skills using machine learning , 2014, KDD.

[15]  Tsong Yueh Chen,et al.  A new heuristic for test suite reduction , 1998, Inf. Softw. Technol..