This paper proposed a new algorithm of inter-frame filtering in IR image based on threshold value for the purpose of solving image blur and smear brought by traditional inter-frame filtering algorithm. At first, it finds out causes of image blur and smear by analyzing general inter-frame filtering algorithm and dynamic inter-frame filtering algorithm, hence to bring up a new kind of time-domain filter. In order to obtain coefficients of the filter, it firstly gets difference image of present image and previous image, and then, it gets noisy threshold value by analyzing difference image with probability analysis method. The relationship between difference image and threshold value helps obtaining the coefficients of filter. At last, inter-frame filtering method is adopted to process pixels interrupted by noise. The experimental result shows that this algorithm has successfully repressed IR image blur and smear, and NETD tested by traditional inter filtering algorithm and the new algorithm are respectively 78mK and 70mK, which shows it has a better noise reduction performance than traditional ones. The algorithm is not only applied to still image, but also to sports image. As a new algorithm with great practical value, it is easy to achieve on FPGA, of excellent real-time performance and it effectively extends application scope of time domain filtering algorithm.