Background-free quantum frequency conversion of single photons from a semiconductor quantum dot