Crowdsourcing System for Multi-object Annotation in Surveillance Videos

The collection and labeling of data is a labor-intensive task and this has given rise to a large market for data crowdsourcing transactions. While there are many publicly available video datasets, task-specific data is still scarce and requires Customized annotation services are required. Even with many excellent auxiliary models and tools, video annotation is still a lengthy and time-consuming task. To address these challenges, this paper provides a new and effective annotation method in which the annotator no longer just provides annotations, but also plays the role of a reviewer to review the annotation results of other annotators. This method focuses on surveillance video data, in addition, it also supports adding additional custom tasks (e.g., action tagging, person relationship recognition, video summarization, etc.). And in this paper we mainly consider the additional custom temporal action annotation task. In this paper, we develop rules for filtering frames or segments that need to be re-labeled based on the temporal information of the model inference results and rely on the correlation between target and time to determine the task relevance, and asynchronously assign the task to different annotators for and dynamically portray the ability of the annotators while annotation is in progress, so as to allocate tasks to achieve annotation and mutual review of annotators. We have experimentally demonstrated that this method can reduce costs and improve labeling accuracy.