Spatio-spectral analysis of ionization times in high-harmonic generation