By performing a full analysis of the projected local density of states (LDOS) in a photonic crystal waveguide, we show that phase plays a crucial role in the symmetry of the light-matter interaction. By considering a quantum dot (QD) spin coupled to a photonic crystal waveguide (PCW) mode, we demonstrate that the light-matter interaction can be asymmetric, leading to unidirectional emission and a deterministic entangled photon source. Further we show that understanding the phase associated with both the LDOS and the QD spin is essential for a range of devices that can be realized with a QD in a PCW. We also show how suppression of quantum interference prevents dipole induced reflection in the waveguide, and highlight a fundamental breakdown of the semiclassical dipole approximation for describing light-matter interactions in these spin dependent systems.