A PAC Framework for Aggregating Agents' Judgments

Specifying the objective function that an AI system should pursue can be challenging. Especially when the decisions to be made by the system have a moral component, input from multiple stakeholders is often required. We consider approaches that query them about their judgments in individual examples, and then aggregate these judgments into a general policy. We propose a formal learning-theoretic framework for this setting. We then give general results on how to translate classical results from PAC learning into results in our framework. Subsequently, we show that in some settings, better results can be obtained by working directly in our framework. Finally, we discuss how our model can be extended in a variety of ways for future research.

[1]  P. Mongin.  Judgment Aggregation , 2011 .

[2]  Nagarajan Natarajan,et al.  Learning with Noisy Labels , 2013, NIPS.

[3]  Koby Crammer,et al.  Learning from Multiple Sources , 2006, NIPS.

[4]  Vincent Conitzer,et al.  Moral Decision Making Frameworks for Artificial Intelligence , 2017, ISAIM.

[5]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[6]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[7]  Kamesh Munagala,et al.  Sequential Deliberation for Social Choice , 2017, WINE.

[8]  Iyad Rahwan,et al.  A Voting-Based System for Ethical Decision Making , 2017, AAAI.

[9]  Vincent Conitzer,et al.  Adapting a Kidney Exchange Algorithm to Align with Human Values , 2018, AAAI.

[10]  Lawrence G. Sager.  Handbook of Computational Social Choice , 2015 .

[11]  Edith Elkind,et al.  Rationalizations of Voting Rules , 2016, Handbook of Computational Social Choice.

[12]  Avrim Blum,et al.  Learning switching concepts , 1992, COLT '92.

[13]  Werner Zellinger,et al.  Moment-Based Domain Adaptation: Learning Bounds and Algorithms , 2020, ArXiv.

[14]  Yishay Mansour,et al.  Domain Adaptation: Learning Bounds and Algorithms , 2009, COLT.

[15]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[16]  Ulrich Endriss,et al.  Complexity of Judgment Aggregation , 2012, J. Artif. Intell. Res..

[17]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[18]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[19]  Koby Crammer,et al.  Learning from Data of Variable Quality , 2005, NIPS.

[20]  Ashish Goel,et al.  Towards Large-Scale Deliberative Decision-Making: Small Groups and the Importance of Triads , 2016, EC.

[21]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[22]  Ariel D. Procaccia,et al.  Collaborative PAC Learning , 2017, NIPS.

[23]  Loizos Michael.  Partial observability and learnability , 2010, Artif. Intell..

[24]  Yishay Mansour,et al.  Domain Adaptation with Multiple Sources , 2008, NIPS.