Automated Formation of Peer-learning Cohorts Using Computer-based Assessment Data: A Double-blind Study within a Software Engineering Course

An approach is developed to integrate the complementary benefits of digitized assessments and peer learning. The research hypothesis is that each student’s assessment data at the fine-grained resolution of correct/incorrect question choice selections can be utilized to partition learners into effective peer learning cohorts. A low overhead approach is explored along with its associated tool, referred to as Automated Peer Learning Cohorts (Auto-PLC). The objective of Auto-PLC is to increase scalability to deliver peer-based learning. First, digitized formative assessments are delivered in a computer-based testing center. This enables automated grading, which frees-up the instructor’s and teaching assistants’ workloads to become reallocated to recitation sessions for higher-gain learning activities, such as peer-based remediation sessions. Second, within the recitations held following each formative quiz, students are afforded an opportunity to complete a remedial assignment. Auto-graded results of formative assessment submissions undergo AutoPLC’s statistical clustering routines using Excel macros and Python scripts to partition the class into four-person peer learning cohorts having mutually-complementary knowledge gaps and skill efficacies. Within each peer learning cohort, students solve together an assigned remedial problem during the recitation session. Thus, students who have already acquired a particular skill become paired together with students who are still acquiring that same skill, and vice versa. This also aids scalability to large enrollments within Electrical and Computer Engineering (ECE) and Computer Science (CS) courses by maximizing opportunities for students to teach each other the material which they still need to learn. The motivation, design, and outcomes for Auto-PLC are presented within the required undergraduate course COP4331: Processes for Object-Oriented Software Development at a large state university. To evaluate effectiveness, a double-blind IRB-approved study has been conducted in COP4331 involving 206 students. All enrolled participated identically, except for their assignment to either randomly-formed or intelligently-clustered remediation groups. At the end of the semester, all students completed an identical Final Exam to provide a basis by which to compare their relative achievements. The data collected expounds upon the details of AutoPLC’s impact towards achievement on a topic-specific basis. Additionally, learners’ perceptions of digitized assessments and participation in recitation-based peer learning cohorts are discussed.