Analysis of the non-reflecting boundary condition for the time-harmonic electromagnetic wave propagation in waveguides