Local Polynomial Regression

Here we provide additional detail about the polynomial regression scheme described in Section 2.2. We consider the quadratic case, as the linear case is a simple restriction thereof. For each component fj of f , the quadratic regressor is of the form f̃j(θ̂) := aj + b T j θ̂ + 1 2 θ̂ Hj θ̂, where aj ∈ R is a constant term, bj ∈ R d is a linear term, and Hj ∈ R d×d is a symmetric Hessian matrix. Note that aj , bj , and Hj collectively contain M = (d + 2)(d + 1)/2 independent entries for each j. The coordinates θ̂ ∈ R are obtained by shifting and scaling the original parameters θ as follows. Recall that the local regression scheme uses N samples {θ, . . . , θ} drawn from the ball of radius R centered on the point of interest θ, along with the corresponding model evaluations y j = fj(θ ). We assume that the components of θ have already been scaled so that they are of comparable magnitudes, then define θ̂ = (θ − θ)/R, so that the transformed samples are centered at zero and have maximum radius one. Writing the error bounds as in (1) requires this rescaling along with the 1/2 in the form of the regressor above (Conn et al., 2009). Next, construct the diagonal weight matrix W = diag(w, . . . , w ) using the sample weights in (2), where we have R = 1 because of the rescaling. Then compute the N -by-M basis matrix Φ: