Nonreciprocal Single-Photon Band Structure.