The effect of singular potentials on the harmonic oscillator