Restricted maximum likelihood estimation of covariances in sparse linear models

This paper surveys the theoretical and computational development of the restricted maximum likelihood (REML) approach for the estimation of covariance matrices in linear stochastic models. A new derivation of this approach is given, valid under very weak conditions on the noise. Then the calculation of the gradient of restricted loglikelihood functions is dis- cussed, with special emphasis on the case of large and sparse model equations with a large number of unknown covariance components and possibly incomplete data. It turns out that the gradient calculations require hardly any extra storage, and only a small multiple of the number of operations needed to calculate the function values alone. The analytic gradient procedure was integrated into the VCE package for co- variance component estimation in large animal breeding models. It resulted in dramatic improvements of performance over the previous implementation with finite difference gradients. An example with more than 250 000 normal equations and 55 covariance components took hours instead of days of CPU time, and this was not an untypical case.