LCrowdV: Generating labeled videos for pedestrian detectors training and crowd behavior learning

Abstract We present a novel procedural framework to generate an arbitrary number of labeled crowd videos (LCrowdV). The resulting crowd video datasets are used to design accurate algorithms or training models for crowded scene understanding. Our overall approach is composed of two components: a procedural simulation framework to generate crowd movements and behaviors, and a procedural rendering framework to generate different videos or images. Each video or image is automatically labeled based on the environment, number of pedestrians, density, behavior (personality), flow, lighting conditions, viewpoint, type of noise, etc. Furthermore, we can increase the realism by combining synthetically-generated behaviors with real-world background videos. We demonstrate the benefits of LCrowdV over prior lableled crowd datasets by augmenting a real dataset with it and improving the accuracy in pedestrian detection and crowd classification. Furthermore, we evaluate the impact of removing the variety in different LCrowdV parameters to show the importance of the diversity of data generated from our framework. LCrowdV has been made available as an online resource.