In this communication, a trainable theory-guided recurrent neural network (RNN) equivalent to the finite-difference-time-domain (FDTD) method is exploited to formulate electromagnetic propagation, solve Maxwell’s equations, and the inverse problem on differentiable programming platform Pytorch. For forward modeling, the computation efficiency is substantially improved compared to conventional FDTD implemented on MATLAB. Gradient computation becomes more precise and faster than the traditional finite difference method benefiting from the accurate and efficient automatic differentiation on the differentiable programming platform. Moreover, by setting the trainable weights of RNN as the material-related parameters, an inverse problem can be solved by training the network. Numerical results demonstrate the effectiveness and efficiency of the method for forward and inverse electromagnetic modeling.