Peer Reviewing in Participatory Guarantee Systems: Modelisation and Algorithmic Aspects

The term Participatory Guarantee Systems (PGS) refers to quality certification systems based on the active participation of stakeholders, i.e., producers, consumers, and experts. Unlike to the more common Third Party Certification system, quality standards are guaranteed by peer review: visits of production sites by producers themselves. A critical issue in PGS is the assignment of the peers carrying each review visit, in a way that incentivizes participation. This paper explores algorithmic aspects of this peer assignment, so as to better address challenges faced by PGS. First, we propose a mathematical model of this task that can express diverse local PGS situations, as well as possible extensions. Then, we show that this model leads to computationally challenging problems and identify restrictions that are easy to handle. Finally, we develop an encoding of the model in Answer Set Programming and use it to solve realistic scenarios of PGS.

[1]  Omer Lev,et al.  Impartial Peer Review , 2015, IJCAI.

[2]  Vincenzina Caputo,et al.  Alternative Labeling Programs and Purchasing Behavior toward Organic Foods: The Case of the Participatory Guarantee Systems in Brazil , 2015 .

[3]  Naren Ramakrishnan,et al.  Recommender systems for the conference paper assignment problem , 2009, RecSys '09.

[4]  Paolo Toth,et al.  Referee Assignment Problem Case: Italian Volleyball Championships , 2012 .

[5]  Martin Gebser,et al.  Modeling and Language Extensions , 2016, AI Mag..

[6]  Nihar B. Shah,et al.  PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review , 2018, ALT.

[7]  Peter A. Flach,et al.  Computational support for academic peer review , 2017, Commun. ACM.

[8]  S. Lemeilleur,et al.  Participatory Guarantee Systems for organic farming: reclaiming the commons , 2019 .

[9]  David G. Kirkpatrick,et al.  On Restricted Two-Factors , 1988, SIAM J. Discret. Math..

[10]  R. Ugás,et al.  Preliminary Results Of The Global Comparative Study On Interactions Between PGS and Social Processes , 2014 .

[11]  C. Vogl,et al.  Participatory Guarantee Systems (PGS) in Mexico: a theoretic ideal or everyday practice? , 2018 .

[12]  Michael D. Plummer,et al.  Graph factors and factorization: 1985-2003: A survey , 2007, Discret. Math..

[13]  Toyohide Watanabe,et al.  Automatic Paper-to-reviewer Assignment, based on the Matching Degree of the Reviewers , 2013, KES.

[14]  Sylvaine Lemeilleur,et al.  Système participatif de garantie dans les labels du mouvement de l’agriculture biologique. Une réappropriation des communs intellectuels , 2018, Économie rurale.

[15]  Martin Gebser,et al.  GrinGo : A New Grounder for Answer Set Programming , 2007, LPNMR.

[16]  Judy Goldsmith,et al.  The AI conference paper assignment problem , 2007, AAAI 2007.

[17]  Igor Labutov,et al.  JAG: A Crowdsourcing Framework for Joint Assessment and Peer Grading , 2017, AAAI.

[18]  Ioannis Caragiannis,et al.  Aggregating Partial Rankings with Applications to Peer Grading in Massive Online Open Courses , 2014, AAMAS.

[19]  Amina Lamghari,et al.  Metaheuristic methods based on Tabu search for assigning judges to competitions , 2010, Ann. Oper. Res..

[20]  Henk Meijer,et al.  An algorithm for computing simple k-factors , 2009, Inf. Process. Lett..

[21]  Martin Gebser,et al.  Answer Set Solving in Practice , 2012, Answer Set Solving in Practice.

[22]  L. Bellante.  Building the local food movement in Chiapas, Mexico: rationales, benefits, and limitations , 2017 .

[23]  Craig Boutilier,et al.  A Framework for Optimizing Paper Matching , 2011, UAI.

[24]  D. Carless,et al.  Peer feedback: the learning element of peer assessment , 2006 .

[25]  Erin M. Nelson,et al.  Participatory organic certification in Mexico: an alternative approach to maintaining the integrity of the organic label , 2010 .

[26]  Erin M. Nelson,et al.  Participatory guarantee systems and the re-imagining of Mexico’s organic sector , 2016 .

[27]  Guillermo Durán,et al.  Referee assignment in the Chilean football league using integer programming and patterns , 2014, Int. Trans. Oper. Res..

[28]  Nancy Falchikov,et al.  Improving Assessment through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education , 2005 .

[29]  John Willmer Escobar,et al.  A flexible mathematical model for the planning and designing of a sporting fixture by considering the assignment of referees , 2019 .

[30]  Justin Cheng,et al.  Peer and self assessment in massive online classes , 2013, ACM Trans. Comput. Hum. Interact..

[31]  Martin Gebser,et al.  clasp : A Conflict-Driven Answer Set Solver , 2007, LPNMR.