Online vs. face-to-face pedagogical code reviews: an empirical comparison

Given the increased importance of communication, teamwork, and critical thinking skills in the computing profession, we have been exploring studio-based instructional methods, in which students develop solutions and iteratively refine them through critical review by their peers and instructor. We have developed an adaptation of studio-based instruction for computing education called the pedagogical code review (PCR), which is modeled after the code inspection process used in the software industry. Unfortunately, PCRs are time-intensive, making them difficult to implement within a typical computing course. To address this issue, we have developed an online environment that allows PCRs to take place asynchronously outside of class. We conducted an empirical study that compared a CS 1 course with online PCRs against a CS 1 course with face-to-face PCRs. Our study had three key results: (a) in the course with face-to-face PCRs, student attitudes with respect to self-efficacy and peer learning were significantly higher; (b) in the course with face-to-face PCRs, students identified more substantive issues in their reviews; and (c) in the course with face-to-face PCRs, students were generally more positive about the value of PCRs. In light of our findings, we recommend specific ways online PCRs can be better designed.

[1]  Karl Wiegers Improving quality with software inspections , 1995 .

[2]  Christopher D. Hundhausen,et al.  The design of an online environment to support pedagogical code reviews , 2010, SIGCSE.

[3]  Daniel D. Suthers,et al.  Kukakuka: An Online Environment for Artifact-Centered Discourse. , 2002 .

[4]  Christopher D. Hundhausen,et al.  Exploring studio-based instructional models for computing education , 2008, SIGCSE '08.

[5]  Christopher D. Hundhausen,et al.  Integrating pedagogical code reviews into a CS 1 course: an empirical study , 2009, SIGCSE '09.

[6]  Dulal C. Kar,et al.  Automatic assignment management and peer evaluation , 2003 .

[7]  Sarah A. Douglas,et al.  Teaching HCI Design With the Studio Approach , 2003, Comput. Sci. Educ..

[8]  Peter J. Middleton,et al.  Software Inspection , 1994, J. Inf. Technol..

[9]  Barbara Means,et al.  Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies , 2009 .

[10]  P. Pintrich A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). , 1991 .

[11]  Karen Swan,et al.  An Investigation into the Efficacy of the Studio Model at the High School Level , 2006 .

[12]  Loren G. Terveen,et al.  Two peers are better than one: aggregating peer reviews for computing assignments is surprisingly accurate , 2009, GROUP.

[13]  Christopher D. Hundhausen,et al.  Does studio-based instruction work in CS 1?: an empirical comparison with a traditional approach , 2010, SIGCSE.

[14]  John H. Schweitzer,et al.  The College Classroom as a Community: Impact on Student Attitudes and Learning , 2006 .

[15]  Margot Brereton,et al.  An innovative design and studio-based CS degree , 2001, SIGCSE '01.

[16]  Edward F. Gehringer,et al.  Electronic peer review and peer grading in computer-science courses , 2001, SIGCSE '01.

[17]  Karen Anewalt Using peer review as a vehicle for communication skill development and active learning , 2005 .

[18]  Deborah Anne Trytten,et al.  A design for team peer code review , 2005, SIGCSE '05.