Measurement of the mechanical reservoir spectral density in an optomechanical system