Towards optimal single-photon sources from polarized microcavities