Towards a field theoretical understanding of kaonic deuterium: leading order retardation effects