The Complexity of Improperly Learning Large Margin Halfspaces

where 〈w,x〉 is the inner product between the vectors x and w. For the 0-1 transfer function, φ0−1(a) = sgn(a)+1 2 , H becomes the class of halfspaces. We allow any transfer functions that satisfy the following (μ, ) margin condition: max{|φ(a)− φ0−1(a)| : |a| > μ} ≤ . For example, the sigmoid function φsig(a) = 1 1+e−a/σ satisfies the (μ, ) condition if σ ≤ μ/(log(1/ − 1). For an illustration see Figure 1. An improper agnostic learning algorithm, A, receives as input a training set of m i.i.d. samples from D, and returns a classifier (not necessarily from H). The output classifier is a random variable and we denote it by A(m). We use err(A(m)) to denote the expected generalization error of the predictor returned by A, where expectation is with respect to the random choice of the training set. We denote by time(A,m) the expected runtime of the algorithm A when running on a training set of m examples.