On self-directed learning

We study several issues concerning the selfdmected model of learning [G RS93, GS94). In particular, we show that the mistake bound of the self-directed learner cannot be smaller than the mistake bound of the on-line learner [L88, L89] by more than a factor of log IX], where X is the instance space. Concept classes which exhibit such a gap are known. In fact, we prove that such a gap exists even between the mistake bound of the self-directed learner and of a learner which is allowed to choose, off-line, the order in which the elements of X will be presented to it (but not adaptively as is the case in the self-directed model); hence, showing the power of adaptiveness. We also study the relations between the VCdimension of a concept class and its self-directed complexity. We show that for certain concept classes the seIf-directed comp~exity can be arbitrarily high, while their VC-dimension remains a fixed constant. This answers an open problem of [GS94]. Moreover, we show that there are “intersection-closed” concept classes for which the self-directed complexity is 3/2 times the VC-dimension. This disproves a conjecture of [GS94]. *Computer Science Department, Technion, Haifa 32000, Israel. shai@cs.technion .ac.il t Computer Science Department, Tecllnion, Haifa 32000, Israel. t Computer Science Department, Technion, Haifa 32000, Israel. eyalkfdcs.teclmi on.ac.il Permission to m~ke digital/hard copies of all or port of this material without fee is granted provided that the copies are not nmde or distributed for urotit or commercial advantage, the ACM copyright/server not~ce, the title of the publication and its date app-e’ar,-and notice is given that copyright is by permission of the Association for Computing Machinery> Inc. (ACM). To copy otherwise, to republish,to post on servers or to We also prove a lower bound on the product of the number of querves made by a selfdirected algorithm and the number of mistakes it makes. This lower bound indicates that lowering the mistake bound of a self-directed algorithm results in an increase of the number of queries it has to go through.

[1]  David Haussler,et al.  Learning decision trees from random examples , 1988, COLT '88.

[2]  N. Littlestone Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[3]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[4]  Shai Ben-David,et al.  Online learning versus offline learning , 1995, EuroCOLT.

[5]  Avrim Blum,et al.  Separating distribution-free and mistake-bound learning models over the Boolean domain , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[6]  Manfred K. Warmuth,et al.  The Weighted Majority Algorithm , 1994, Inf. Comput..

[7]  Philip M. Long,et al.  Apple tasting and nearly one-sided learning , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[8]  Zhixiang Chen,et al.  On-line learning of rectangles , 1992, COLT '92.

[9]  Ronald L. Rivest,et al.  Learning Binary Relations and Total Orders , 1993, SIAM J. Comput..

[10]  D. Angluin Queries and Concept Learning , 1988 .

[11]  Wolfgang Maass,et al.  On-line learning with an oblivious environment and the power of randomization , 1991, COLT '91.

[12]  P. Gács,et al.  Algorithms , 1992 .

[13]  Avrim Blum Rank-r Decision Trees are a Subclass of r-Decision Lists , 1992, Inf. Process. Lett..