Minimax-optimal rates for high-dimensional sparse additive models over kernel classes

Sparse additive models are families of d-variate functions that have the additive decomposition f = ∑ j∈S f ∗ j , where S is an unknown subset of cardinality s ≪ d. We consider the case where each component function f j lies in a reproducing kernel Hilbert space, and analyze an l1 kernel-based method for estimating the unknown function f . Working within a highdimensional framework that allows both the dimension d and sparsity s to increase with n, we derive rates in the L(P) and L(Pn) norms over the class Fd,s,H of sparse additive models with each univariate function f j bounded. These rates consist of two terms: a subset selection term of the order s log d n , corresponding to the difficulty of finding the unknown s-sized subset, and an estimation error term of the order s ν n, where ν 2 n is the optimal rate for estimating an univariate function within the RKHS. We complement these achievable results by deriving minimax lower bounds on the L(P) error, thereby showing the optimality of our method. Thus, we obtain optimal minimax rates for many interesting classes of sparse additive models, including polynomials, splines, finite-rank kernel classes, as well as Sobolev smoothness classes. Concurrent work by Koltchinskii and Yuan [17] analyzes the same l1-kernel-based estimator, and under an additional global boundedness condition, provides rates on the L(P) and L(Pn) of the same order as those proven here without global boundedness. We analyze an alternative estimator for globally bounded function classes, and prove that it can achieve strictly faster rates for Sobolev smoothness classes with sparsity s = Ω( √ n). Consequently, in the high-dimensional setting, the minimax rates with global boundedness conditions are strictly faster, so that the rates proven by Koltchinskii and Yuan [17] are not minimax optimal for growing sparsity.