Online Segmentation of Continuous Gesturing in Interfaces

An increasing demand exists for more intuitive ways to interact with ever larger displays. Natural interfaces fulfill this demand by directly analyzing, reacting to and reasoning about observed human behavior. We focus on gesturing which is a key modality in a more complex natural interface. Most gesture recognition research today focuses on computer vision techniques. Although promising, such techniques are not yet mature enough for markerless, robust and in-the-field deployment. Instead, we deploy a more down-to-earth solution by using existing motion capture systems. Our goal is to develop methods for automatic online segmentation and interpretation of continuous gesturing. We propose a twostage approach. First, we use commodity hardware equipped with accelerometers and buttons that explicitly mark gesture boundaries. Experience gained there feeds our second stage in which we investigate motion trajectories and their meaning. In the second stage we use a combination of a full body motion capture suit and two data gloves.