All things retarded: radiation-reaction in worldline quantum field theory