This paper deals with vibration-based damage localization and quantification from output-only measurements. We describe an approach which operates on a data-driven residual vector that is statistically evaluated using information from a finite element model, without updating the parameters of the model. First, the damaged elements are detected in statistical tests, and second, the damage is quantified only for the damaged elements. We propose a new residual vector in this context that is based on the transfer matrix difference between reference and damaged states, and compare it with a previously introduced subspace-based residual. We show localization and quantification on both residuals in simulations.