Effect of environment on the interferometry of clocks