A fast ‘Monte-Carlo cross-validation’ procedure for large least squares problems with noisy data

SummaryWe propose a fast Monte-Carlo algorithm for calculating reliable estimates of the trace of the influence matrixAτ involved in regularization of linear equations or data smoothing problems, where τ is the regularization or smoothing parameter. This general algorithm is simply as follows: i) generaten pseudo-random valuesw1, ...,wn, from the standard normal distribution (wheren is the number of data points) and letw=(w1, ...,wn)T, ii) compute the residual vectorw−Aτw, iii) take the ‘normalized” inner-product (wT(w−Aτw))/(wTw) as an approximation to (1/n)tr(I−Aτ). We show, both by theoretical bounds and by numerical simulations on some typical problems, that the expected relative precision of these estimates is very good whenn is large enough, and that they can be used in practice for the minimization with respect to τ of the well known Generalized Cross-Validation (GCV) function. This permits the use of the GCV method for choosing τ in any particular large-scale application, with only a similar amount of work as the standard residual method. Numerical applications of this procedure to optimal spline smoothing in one or two dimensions show its efficiency.