FasterMDNet: Learning model adaptation by RNN in tracking-by-detection based visual tracking

Recently in VOT competitions, trackers based on tracking-by-detection and deep neural network discriminators reached impressive accuracy. However, these trackers require time-consuming model adaptation methods like online learning to handle target appearance changes, which tremendously increases the time complexity and becomes an obstruction in realworld applications for these tracking algorithms. In this paper, we propose an efficient RNN-based model adaptation method which extremely decreases the time complexity of trackers. The proposed model learns the relations of relative model change in RNN training and predicts the score and the model adaptation state at the same time in testing, which nearly removes the finetuning time in the cost of additional RNN training. The proposed method is applicable to any tracker based on neural network discriminator. The RNN branch can be further designed with more complicated model under the condition of enough training videos. We apply the proposed algorithm to MDNet and create a new tracker: Faster-MDNet. According to the experiment, using our method can nearly remove the time of finetuning and reduce the bottleneck of time-complexity down to the prediction time.