Boosting and hard-core set constructions: a simplified approach

We revisit the connection between boosting algorithms and hard-core set constructions discovered by Klivans and Servedio. We present a boosting algorithm with a certain smoothness property that is necessary for hard-core set constructions: the distributions it generates do not put too much weight on any single example. We then use this boosting algorithm to show the existence of hard-core sets matching the best parameters of Klivans and Servedio’s construction. 1 Boosting and Hard-Core sets Hard-core set constructions are a form of hardness amplification of boolean functions. In such constructions, starting with a function that is mildly inapproximable by circuits of a certain size, we obtain a distribution on inputs such that the same function is highly inapproximable by circuits of size closely related to the original, when inputs are drawn from it. Impagliazzo [3] gave the first hard-core set constructions and used them to give an alternative proof of Yao’s XOR lemma. Klivans and Servedio [4] gave improved hard-core set constructions using a connection to the notion of boosting from learning theory. The goal of boosting is to “boost” some small initial advantage over random guessing that a learner can achieve in Valiant’s PAC (Probabilistically Approximately Correct) model of learning. Klivans and Servedio describe how boosting algorithms which satisfy certain smoothness properties can be generically used to obtain good hard-core set constructions. We give a new algorithm for boosting that enjoys the smoothness properties necessary for hardcore set constructions. This algorithm has the additional desirable property that it has the same number of iterations as the AdaBoost algorithm, and thus is more efficient Servedio’s SmoothBoost algorithm [5]. For this reason, the boosting algorithm should be of independent interest, especially for the applications in Servedio’s paper [5] on learning in the presence of malicious noise. Our boosting algorithm is inspired by Warmuth and Kuzmin’s [7] technique (which, in turn, uses ideas that originated in the work Herbster and Warmuth [2]) of obtaining smooth distributions from any other distribution by projecting it into the set of smooth distributions using the relative entropy as a distance function. We use this boosting algorithm to construct hard-core sets matching the best parameters of the Klivans-Servedio construction. Although we do not improve over the parameters of Klivans and Servedio’s construction, our proof is arguably simpler for three reasons: (a) it obtains the best known parameters for hard-core set constructions directly by applying the boosting algorithm, rather than building it up incrementally by accumulating small hard-core sets, and (b) it obviates 1 Electronic Colloquium on Computational Complexity, Report No. 131 (2007)