In-Flow Peer Review

Peer-review is a valuable tool that helps both the reviewee, who receives feedback about his work, and the reviewer, who sees different potential solutions and improves her ability to critique work. In-flow peer-review (IFPR) is peer-review done while an assignment is in progress. Peer-review done during this time is likely to result in greater motivation for both reviewer and reviewee. This workinggroup report summarizes IFPR and discusses numerous dimensions of the process, each of which alleviates some problems while raising associated concerns.

[1]  Christopher D. Hundhausen,et al.  Talking about code: Integrating pedagogical code reviews into early computing courses , 2013, TOCE.

[2]  Eric Mazur,et al.  Peer Instruction: A User's Manual , 1996 .

[3]  Shriram Krishnamurthi.  The CONTINUE Server (or, How I Administered PADL 2002 and 2003) , 2003, PADL.

[4]  Sumit Gulwani,et al.  Automatically Generating Algebra Problems , 2012, AAAI.

[5]  Matthias Hauswirth,et al.  Teaching Java programming with the Informa clicker system , 2013, Sci. Comput. Program..

[6]  Simon Hooper,et al.  The effects of group composition on achievement, interaction, and learning efficiency during computer-based cooperative instruction , 1991 .

[7]  Theodore C. Willoughby.  Review of "The Psychology of Computer Programming, by Gerald M. Weinberg." Van Nostrand Reinhold Company, 1971 , 1972, DATB.

[8]  Gerald M. Weinberg,et al.  Psychology of computer programming , 1971 .

[9]  Justin Cheng,et al.  Peer and self assessment in massive online classes , 2013, ACM Trans. Comput. Hum. Interact..

[10]  John Hamer,et al.  PeerWise: students sharing their multiple choice questions , 2008, ICER '08.

[11]  Judy Kay,et al.  A mentor program in CS1 , 2002, ITiCSE '02.

[12]  Patrick Wessa.  How Reproducible Research Leads to Non-Rote Learning within Socially Constructivist Statistics Education , 2009 .

[13]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[14]  G. Hofstede,et al.  Cultures and Organizations: Software of the Mind , 1991 .

[15]  Fiona Spence,et al.  Peer assessment using aropä , 2007 .

[16]  Edward F. Gehringer,et al.  Automated Assessment of Review Quality Using Latent Semantic Analysis , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[17]  Quincy Brown,et al.  AP® computer science principles , 2015, Inroads.

[18]  C. MacArthur,et al.  Student revision with peer and expert reviewing , 2010 .

[19]  Scott R. Klemmer,et al.  Early and Repeated Exposure to Examples Improves Creative Work , 2012, CogSci.

[20]  Anneli Eteläpelto.  Metacognition and the Expertise of Computer Program Comprehension , 1993 .

[21]  Aytaç Göğüş.  Peer learning and assessment , 2012 .

[22]  R. Brandt.  Punished by Rewards , 1995 .

[23]  Thomas Lagkas,et al.  How to improve the peer review method: Free-selection vs assigned-pair protocol evaluated in a computer networking course , 2012, Comput. Educ..

[24]  Andreas Zeller,et al.  Making students read and review code , 2000, ITiCSE '00.

[25]  Loren G. Terveen,et al.  Two peers are better than one: aggregating peer reviews for computing assignments is surprisingly accurate , 2009, GROUP.

[26]  Simon Hooper,et al.  Cooperative CBI: The Effects of Heterogeneous versus Homogeneous Grouping on the Learning of Progressively Complex Concepts , 1988 .

[27]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[28]  Judithe Sheard,et al.  Contributing student pedagogy , 2012, Comput. Sci. Educ..

[29]  S. Derry,et al.  Learning from Examples: Instructional Principles from the Worked Examples Research , 2000 .

[30]  Nicole Clark.  Peer testing in Software Engineering Projects , 2004 .

[31]  Jean Scholtz,et al.  Mental representations of programs by novices and experts , 1993, INTERCHI.

[32]  D. Carless,et al.  Peer feedback: the learning element of peer assessment , 2006 .

[33]  Quintin I. Cutts,et al.  Student experience in a student-centered peer instruction classroom , 2013, ICER.

[34]  Andrew Luxton-Reilly,et al.  A systematic review of tools that support peer assessment , 2009, Comput. Sci. Educ..

[35]  Christian D. Schunn,et al.  Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system , 2007, Comput. Educ..

[36]  Forrest Shull,et al.  Impact of research on practice in the field of inspections, reviews and walkthroughs: learning from successful industrial uses , 2008, SOEN.

[37]  Joe Gibbs Politz,et al.  CaptainTeach: multi-stage, in-flow peer review for programming assignments , 2014, ITiCSE '14.

[38]  Ursula Fuller,et al.  Perspectives on developing and assessing professional values in computing , 2010, SGCS.

[39]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[40]  N. Black,et al.  Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. , 1999, Journal of clinical epidemiology.

[41]  E. Mazur,et al.  Peer Instruction: Ten years of experience and results , 2001 .

[42]  Jiang Yu,et al.  Learning outcomes of programming language courses based on peer code review model , 2011, 2011 6th International Conference on Computer Science & Education (ICCSE).

[43]  John Hamer,et al.  A Method of Automatic Grade Calibration in Peer Assessment , 2005, ACE.

[44]  Richard van de Stadt.  CyberChair: A Web-Based Groupware Application to Facilitate the Paper Reviewing Process , 2012, ArXiv.

[45]  Jenny C. C. Chung *,et al.  Promoting student learning through a student‐centred problem‐based learning subject curriculum , 2004 .

[46]  K. Garvin-Doxas,et al.  What can computer science learn from a fine arts approach to teaching? , 2005, SIGCSE '05.

[47]  Eddie Kohler.  Hot Crap! , 2008, WOWCS.

[48]  Joe Gibbs Politz,et al.  In-flow peer-review of tests in test-first programming , 2014, ICER '14.

[49]  Alberto Bacchelli,et al.  Expectations, outcomes, and challenges of modern code review , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[50]  Edward F. Gehringer,et al.  Reusable Learning Objects Thr ough Peer Review: The E xpertiza Approach , 2007 .

[51]  Christian D. Schunn,et al.  The nature of feedback: how different types of peer feedback affect writing performance , 2009 .

[52]  Joanna Smith,et al.  Using peer review to teach software testing , 2012, ICER '12.

[53]  Barbara Y. White,et al.  Cognitive facilitation: a method for promoting reflective collaboration , 1997, CSCL.

[54]  L. Vygotsky.  Interaction between learning and development , 1978 .

[55]  Nancy M. Trautmann.  Designing Peer Review for Pedagogical Success: What Can We Learn from Professional Science?. , 2009 .

[56]  A. DeRycker.  Reviewing Peer Reviews: A Rule-Based Approach , 2010 .

[57]  Mark Guzdial,et al.  Situating CoWeb: a scholarship of application , 2006, Int. J. Comput. Support. Collab. Learn..

[58]  Ann L. Brown,et al.  Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities , 1984 .

[59]  Lecia Jane Barker,et al.  Communication in computer science classrooms: understanding defensive climates as a means of creating supportive behaviors , 2004, JERC.

[60]  G. Sims,et al.  Student Peer Review in the Classroom: A Teaching and Grading Tool. , 1989 .

[61]  J. Palme,et al.  Computer-mediated communication in collaborative educational settings: report of the ITiCSE '97 working group on CMC in collaborative educational settings , 1997, SCOU.

[62]  Tony Clear.  THINKING ISSUESManaging mid-project progress reviews: a model for formative group assessment in capstone projects , 2010, INROADS.

[63]  K. Topping.  Peer Assessment Between Students in Colleges and Universities , 1998 .

[64]  Harald Søndergaard,et al.  Learning from and with peers: the different roles of student peer reviewing , 2009, ITiCSE.

[65]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[66]  Elizabeth A. Davis,et al.  Scaffolding students' knowledge integration: prompts for reflection in KIE , 2000 .

[67]  Cynthia Bailey Lee,et al.  Can peer instruction be effective in upper-division computer science courses? , 2013, TOCE.

[68]  Tony Clear,et al.  An introduction to program comprehension for computer science educators , 2010, ITiCSE-WGR '10.

[69]  Andrew Luxton-Reilly,et al.  A comparison of peer and tutor feedback , 2015 .

[70]  Sumit Gulwani,et al.  Synthesis of Geometry Proof Problems , 2014, AAAI.