Vision based robotic gesture recognition

Vision in complex environments is a big scientific challenge for two reasons. First, in complex environments it is not possible to segment a scene into the constituent objects on the basis of simple cues. Second, unpredictable changes in the environment must be tolerated. A proper domain for investigating these problems is robotic gesture recognition, since the two problems arise there naturally. Furthermore, gesture recognition holds the promise of making man-machine interaction more natural and intuitive. The principal idea for tackling the first problem is the integration of information stemming from different cues. In the first part of this thesis, methods for tracking human hands, finding fingertips and recognizing hand postures despite complex backgrounds are presented, which owe their robustness to the integration of different complementary cues. The components have been integrated into a user-independent gesture interface implemented on an anthropomorphic robot. The second part is concerned with the adaptive integration of different cues, aimed at addressing the second problem. A model of adaptive sensory integration in the brain is proposed, which relates the psychophysical phenomena of suppression and recalibration of discordant sensory information to a self-organized adaptation employing fast synaptic plasticity mechanisms. Finally, the idea of self-organized adaptation is applied to the tracking of human faces in a scene. To this end, an adaptive tracking scheme is proposed which combines different cues in a “democratic” manner.