Towards Automatic Discovery of Human Movemes

Consider a number of moving points, each attached to a joint of the human body and projected onto an image. Johannson showed that humans can effortlessly detect and recognize the presence of other humans from such displays. This is true even when some of the body parts are missing (e.g., because of occlusion) and unrelated clutter points are added to the display. Furthermore, subtle aspects like age range and gender, as well as the ongoing activity, can be inferred with a surprising degree of accuracy from such a seemingly scarce amount of information. We are interested in replicating some of these abilities in a machine. We start by introducing a labeling and detection scheme in a Johannson-like display. Our method is based on a probabilistic representation of the positions and motion of body parts, which we use to calculate a likely interpretation of the scene by means of belief propagation techniques. We show how learning and inference can be done efficiently, and we provide an experimental validation of the method. In the second part of our work, we present our position on the analysis of human behaviors. We hypothesize a hierarchical description of motion, which provides a natural interpretation of actions and activities as stochastic sequences of "atomic motions" or movemes. We take an initial step in that direction by illustrating how to learn a dictionary of movemes from the trajectories of body parts, which can be used to concisely represent the video for further analysis.