PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance

Rapid feedback is a core component of mastery learning, but feedback on open-ended work requires days or weeks in most classes today. This paper introduces PeerStudio, an assessment platform that leverages the large number of students' peers in online classes to enable rapid feedback on in-progress work. Students submit their draft, give rubric-based feedback on two peers' drafts, and then receive peer feedback. Students can integrate the feedback and repeat this process as often as they desire. In MOOC deployments, the median student received feedback in just twenty minutes. Rapid feedback on in-progress work improves course outcomes: in a controlled experiment, students' final grades improved when feedback was delivered quickly, but not if delayed by 24 hours. More than 3,600 students have used PeerStudio in eight classes, both massive and in-person. This research demonstrates how large classes can leverage their scale to encourage mastery through rapid feedback and revision.

[1]  Sumit Basu,et al.  Divide and correct: using clusters to grade short answers at scale , 2014, L@S.

[2]  Michael S. Bernstein,et al.  Who gives a tweet?: evaluating microblog content value , 2012, CSCW.

[3]  Scott R. Klemmer,et al.  The efficacy of prototyping under time constraints , 2009, C&C '09.

[4]  Michael S. Bernstein,et al.  Crowd-scale interactive formal reasoning and analytics , 2013, UIST.

[5]  J. Krosnick,et al.  Survey research. , 1999, Annual review of psychology.

[6]  Daniel L. Schwartz,et al.  Prototyping dynamics: sharing multiple designs improves exploration, group rapport, and results , 2011, CHI.

[7]  Wayne D. Gray,et al.  Milliseconds Matter: an Introduction to Microstrategies and to Their Use in Describing and Predicting Interactive Behavior Milliseconds Matter: an Introduction to Microstrategies and to Their Use in Describing and Predicting Interactive Behavior , 2022 .

[8]  T. Guskey.  Closing Achievement Gaps: Revisiting Benjamin S. Bloom's “Learning for Mastery” , 2007 .

[9]  P. Carlson,et al.  TM AND ASSESSING LEARNING OUTCOMES , 2003 .

[10]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[11]  Bill Buxton,et al.  Sketching User Experiences: Getting the Design Right and the Right Design , 2007 .

[12]  N. Sommers.  Responding to Student Writing , 1982, College Composition & Communication.

[13]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[14]  H. Andrade.  The Effects of Instructional Rubrics on Learning to Write , 2001 .

[15]  R. Marsh,et al.  How examples may (and may not) constrain creativity , 1996, Memory & cognition.

[16]  P. A. Carlson,et al.  Calibrated peer review/sup TM/ and assessing learning outcomes , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[17]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[18]  Loren Olson,et al.  CritViz: Web-Based Software Supporting Peer Critique in Large Creative Classrooms , 2013 .

[19]  Justin Cheng,et al.  Peer and self assessment in massive online classes , 2013, ACM Trans. Comput. Hum. Interact..

[20]  Michael S. Bernstein,et al.  Scaling short-answer grading by combining peer assessment with algorithmic scoring , 2014, L@S.

[21]  H. Andrade.  Teaching With Rubrics: The Good, the Bad, and the Ugly , 2005 .

[22]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[23]  E. A. Locke,et al.  Self-regulation through goal setting , 1991 .

[24]  Chen-Lin C. Kulik,et al.  Timing of Feedback and Verbal Learning , 1988 .

[25]  Fabricio E. Balcazar,et al.  A Critical, Objective Review of Performance Feedback , 1985 .

[26]  J. Rodin,et al.  Is bad news always bad? Cue and feedback effects on intrinsic motivation. , 1989 .

[27]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[28]  Soo-Min Kim,et al.  Automatically Assessing Review Helpfulness , 2006, EMNLP.

[29]  Ivan Poupyrev,et al.  Proceedings of the 26th annual ACM symposium on User interface software and technology , 2013, UIST 2013.

[30]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.