Interactive tracking of insect posture

In this paper, we present an association based tracking approach to track multiple insect body parts in a set of low frame-rate videos. The association is formulated as a MAP problem and solved by the Hungarian algorithm. Different from a traditional track-and-then-rectification scheme, this framework refines the tracking hypotheses in an interactive fashion: it integrates a key frame selection approach to minimize the number of frames for user correction while optimizing the final hypotheses. Given user correction, it takes user inputs to rectify the incorrect hypotheses on the other frames. Thus, the framework improves the tracking accuracy by introducing active key frame selection and interactive components, enabling a flexible strategy to achieve a trade-off between human effort and tracking precision. Given the refined tracks at a bounding box (BB) level, the tip of each body part is estimated, and multiple body parts in a BB are further differentiated. The efficiency and the effectiveness of the framework are verified on challenging video datasets for insect behavioral experiments. HighlightsGeneral interactive tracking framework for behavioral analysis in videos.Rectification of incorrect hypotheses incorporating the initial information.Active key frame selection based on the estimation of annotation costs.