Multi-View Active Learning in the Non-Realizable Case

The sample complexity of active learning under the realizability assumption has been well-studied. The realizability assumption, however, rarely holds in practice. In this paper, we theoretically characterize the sample complexity of active learning in the non-realizable case under multi-view setting. We prove that, with unbounded Tsybakov noise, the sample complexity of multi-view active learning can be O(log 1/e), contrasting to single-view setting where the polynomial improvement is the best possible achievement. We also prove that in general multi-view setting the sample complexity of active learning with unbounded Tsybakov noise is O(1/e), where the order of 1/e is independent of the parameter in Tsybakov noise, contrasting to previous polynomial bounds where the order of 1/e is related to the parameter in Tsybakov noise.

[1]  Maria-Florina Balcan,et al.  The true sample complexity of active learning , 2010, Machine Learning.

[2]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[3]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[4]  John Langford,et al.  Agnostic active learning , 2006, J. Comput. Syst. Sci..

[5]  Robert D. Nowak,et al.  Minimax Bounds for Active Learning , 2007, IEEE Transactions on Information Theory.

[6]  Sanjoy Dasgupta,et al.  Coarse sample complexity bounds for active learning , 2005, NIPS.

[7]  Steve Hanneke,et al.  Adaptive Rates of Convergence in Active Learning , 2009, COLT.

[8]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[9]  Adam Tauman Kalai,et al.  Analysis of Perceptron-Based Active Learning , 2009, COLT.

[10]  A. Tsybakov,et al.  Optimal aggregation of classifiers in statistical learning , 2003 .

[11]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[12]  Matti Kääriäinen,et al.  Active Learning in the Non-realizable Case , 2006, ALT.

[13]  Sanjoy Dasgupta,et al.  Analysis of a greedy active learning strategy , 2004, NIPS.

[14]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[15]  Sanjoy Dasgupta,et al.  A General Agnostic Active Learning Algorithm , 2007, ISAIM.

[16]  R. Nowak,et al.  Upper and Lower Error Bounds for Active Learning , 2006 .

[17]  Steve Hanneke,et al.  A bound on the label complexity of agnostic active learning , 2007, ICML '07.

[18]  Zhi-Hua Zhou,et al.  On multi-view active learning and the combination with semi-supervised learning , 2008, ICML '08.

[19]  Liwei Wang,et al.  Sufficient Conditions for Agnostic Active Learnable , 2009, NIPS.

[20]  Maria-Florina Balcan,et al.  Margin Based Active Learning , 2007, COLT.

[21]  Claudio Gentile,et al.  Linear Classification and Selective Sampling Under Low Noise Conditions , 2008, NIPS.

[22]  Peter L. Bartlett,et al.  Learning in Neural Networks: Theoretical Foundations , 1999 .

[23]  Maria-Florina Balcan,et al.  Co-Training and Expansion: Towards Bridging Theory and Practice , 2004, NIPS.

[24]  Craig A. Knoblock,et al.  Active + Semi-supervised Learning = Robust Multi-View Learning , 2002, ICML.

[25]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.