Human Body Posture via Hierarchical Evolutionary Optimization

This paper presents an evolutionary approach to estimating upper-body posture from multi-view markerless sequences. We fit a 24-dof skeleton model to sparse 3-D stereo data from an array of cameras. We use a particle swarm optimization algorithm which is intrinsically parallel, can incorporate constraints and does not require motion models. We subdivide the high-dimensional search space based on limb dynamics from application sequences and perform hierarchical fitting from the least to the most uncertain body parts. We show experimentally the advantages of this scheme against non-hierarchical optimization in terms of sharper error decrease. We report results with 3-D scanner data of a model human and noisy, calibrated stereo disparity maps of a real videoconferencing scene. 1 Introduction and motivation This paper presents an evolutionary approach to estimating upper-body posture from multi-view markerless sequences. Our reference application is immersive videoconferencing, which aims to create an impression of presence, or co-location, among a group of participants situated at different geographical locations but meeting in a common, augmented reality space [17, 18, 19]. Vital for presence are 3-D visual cues, e.g., rendering figures consistently with the instantaneous viewer’s viewpoint. This requires estimates of the 3-D structure of the scene, which is typically done by multiview disparity analysis (see [1] and references above). Disparity maps (DMs) allow IBR-oriented systems to perform novel view synthesis consistent with the target viewpoint. A major problem is to achieve in real time DMs of the moving human body with sufficiently high quality, as typical immersive environments use large screens. Notice that videoconferencing setups rarely allow high numbers of cameras surrounding the scene [2]; occlusions created by gestures make DMs sparse. Assuming suitable computing power, integrating bottom-up disparities with body models can yield superior quality DMs. Previously, only local DM enhancement has been attempted by infilling [3, 4]. In order to further this idea, we intend to fit human body models to DMs of multi-view videoconferencing sequences. This paper concentrates on the first step, skeleton fitting for posture estimation, and introduces a novel evolutionary approach to the high-dimensional optimization problems typical of body model fitting with markerless sequences. 1