Effects of Network Topology on the OpenAnswer's Bayesian Model of Peer Assessment

The paper investigates if and how the topology of the peer-assessment network can affect the performance of the Bayesian model adopted in OpenAnswer. Performance is evaluated in terms of the comparison of predicted grades with actual teacher’s grades. The global network is built by interconnecting smaller subnetworks, one for each student, where intra-subnetwork nodes represent student’s characteristics, and peer assessment assignments make up inter-subnetwork connections and determine evidence propagation. A possible subset of teacher graded answers is dynamically determined by suitable selection and stop rules. The research questions addressed are: (RQ1) “does the topology (diameter) of the network negatively influence the precision of predicted grades?”; in the affirmative case, (RQ2) “are we able to reduce the negative effects of high-diameter networks through an appropriate choice of the subset of students to be corrected by the teacher?” We show that (RQ1) OpenAnswer is less effective on higher diameter topologies, (RQ2) this can be avoided if the subset of corrected students is chosen considering the network topology.