List Learning with Attribute Noise

We introduce and study the model of list learning with attribute noise. Learning with attribute noise was introduced by Shackelford and Volper (COLT 1988) as a variant of PAC learning, in which the algorithm has access to noisy examples and uncorrupted labels, and the goal is to recover an accurate hypothesis. Sloan (COLT 1988) and Goldman and Sloan (Algorithmica 1995) discovered information-theoretic limits to learning in this model, which have impeded further progress. In this article we extend the model to that of list learning, drawing inspiration from the list-decoding model in coding theory, and its recent variant studied in the context of learning. On the positive side, we show that sparse conjunctions can be efficiently list learned under some assumptions on the underlying ground-truth distribution. On the negative side, our results show that even in the list-learning model, efficient learning of parities and majorities is not possible regardless of the representation used.

[1]  Peter Elias,et al.  List decoding for noisy channels , 1957 .

[2]  D. Angluin,et al.  Learning From Noisy Examples , 1988, Machine Learning.

[3]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[4]  George Shackelford,et al.  Learning k-DNF with noise in the attributes , 1988, Annual Conference Computational Learning Theory.

[5]  Rosario Gennaro,et al.  On learning from noisy and incomplete examples , 1995, COLT '95.

[6]  Sally A. Goldman,et al.  Can PAC learning algorithms tolerate random attribute noise? , 1995, Algorithmica.

[7]  Santosh S. Vempala,et al.  A discriminative framework for clustering via similarity functions , 2008, STOC.

[8]  Amin Karbasi,et al.  Group Testing With Probabilistic Tests: Theory, Design and Application , 2010, IEEE Transactions on Information Theory.

[9]  Loizos Michael Partial observability and learnability , 2010, Artif. Intell..

[10]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[11]  Robert H. Sloan,et al.  Corrigendum to types of noise in data for concept learning , 1988, COLT '92.

[12]  D. Rubin INFERENCE AND MISSING DATA , 1975 .

[13]  T. Sanders Analysis of Boolean Functions , 2012, ArXiv.

[14]  Daniel M. Kane,et al.  List-decodable robust mean estimation and learning mixtures of spherical gaussians , 2017, STOC.

[15]  Dale Schuurmans,et al.  Learning Default Concepts , 1994 .

[16]  Prasad Raghavendra,et al.  List Decodable Learning via Sum of Squares , 2019, SODA.

[17]  Nader H. Bshouty,et al.  Uniform-distribution attribute noise learnability , 1999, COLT '99.

[18]  Ming Li,et al.  Learning in the presence of malicious errors , 1993, STOC '88.

[19]  Gregory Valiant,et al.  Learning from untrusted data , 2016, STOC.

[20]  Adam R. Klivans,et al.  List-Decodable Linear Regression , 2019, NeurIPS.

[21]  Vladimir M. Blinovsky,et al.  List decoding , 1992, Discrete Mathematics.