AutoStyle: Scale-driven Hint Generation for Coding Style

While the use of autograders for code correctness is widespread, less e↵ort has focused on automating feedback for good programming style: the tasteful use of language features and idioms to produce code that is not only correct, but also concise, elegant, and revealing of design intent. We present a system that can provide real-time actionable code style feedback to students in large introductory computer science classes. We demonstrate that in a randomized controlled trial, 70% of students using our system achieved the best style solution to a coding problem in less than an hour, while only 13% of students in the control group achieved the same. Students using our system also showed a statisticallysignificant greater improvement in code style than students in the control group. We also present experiments to demonstrate the e cacy and relevance of each of the di↵erent types of hints generated by our system.

[1]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[2]  Mauricio A. Saca.  Refactoring improving the design of existing code , 2017, 2017 IEEE 37th Central America and Panama Convention (CONCAPAN XXXVII).

[3]  Kenneth R. Koedinger,et al.  Data-Driven Hint Generation in Vast Solution Spaces: a Self-Improving Python Programming Tutor , 2015, International Journal of Artificial Intelligence in Education.

[4]  Armando Fox,et al.  AutoStyle: Toward Coding Style Feedback At Scale , 2016, CSCW Companion.

[5]  Armando Fox,et al.  Clustering Student Programming Assignments to Multiply Instructor Leverage , 2015, L@S.

[6]  Michael S. Bernstein,et al.  PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance , 2015, L@S.

[7]  Ivan Bratko,et al.  Data-Driven Program Synthesis for Hint Generation in Programming Tutors , 2014, Intelligent Tutoring Systems.

[8]  Michael S. Bernstein,et al.  Emergent, crowd-scale programming practice in the IDE , 2014, CHI.

[9]  Elena L. Glassman,et al.  Feature engineering for clustering student solutions , 2014, L@S.

[10]  Robert C. Martin.  Clean Code - a Handbook of Agile Software Craftsmanship , 2008 .

[11]  T. Guskey.  Closing Achievement Gaps: Revisiting Benjamin S. Bloom's “Learning for Mastery” , 2007 .

[12]  V. Shute.  Focus on Formative Feedback , 2007 .

[13]  Stephen H. Kan,et al.  Metrics and Models in Software Quality Engineering , 1994, SOEN.

[14]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[15]  Kaizhong Zhang,et al.  Simple Fast Algorithms for the Editing Distance Between Trees and Related Problems , 1989, SIAM J. Comput..

[16]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[17]  David A. Patterson,et al.  MAGIC: Massive Automated Grading in the Cloud , 2015, CHANGEE/WAPLA/HybridEd@EC-TEL.

[18]  Leonidas J. Guibas,et al.  Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC , 2013, AIED Workshops.

[19]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[20]  Jerry Fitzpatrick,et al.  Applying the ABC metric to C, C++, and Java , 2000 .

[21]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[22]  Murray Hill,et al.  Lint, a C Program Checker , 1978 .