Abstract Target tracking under microscopic spectrum is a hot topic in computer vision. Because of its motion characteristics, there are some problems, such as small area dither, segmentation difficulty and so on, an improved tracking algorithm for multiple abnormal targets of micro spectral image is proposed in this paper. Based on acquiring the gradient vector flow in the dithering region of image, the original images in the process of multiple abnormal targets tracking are reconstructed. Through the motion process deduction, the more accurate curve of two-dimensional image composed of feature points of image segmentation boundary is obtained. In the process of tracking the moving target contour in the dither state, the parameter model of the Gauss distribution of the moving image background is constructed by integrating the statistical theory. The experimental results show that the improved tracking algorithm can effectively improve the robustness of the tracking algorithm.