Robust recognition of physical team behaviors using spatio-temporal models

This paper presents a framework for robustly recognizing physical team behaviors by exploiting spatio-temporal patterns. Agent team behaviors in athletic and military domains typically exhibit an observable structure characterized by the relative positions of teammates and external landmarks, such as a team of soldiers ambushing an opponent or a soccer player moving to receive a pass. We demonstrate how complex team relationships that are not easily expressed by region-based heuristics can be modeled from data and domain knowledge in a way that is robust to noise and spatial variation. To represent team behaviors in our domain of MOUT (Military Operations in Urban Terrain) planning, we employ two classes of spatial models: 1) team templates that encode static relationships between team members and external landmarks; and 2) spatially-invariant Hidden Markov Models (HMMs) to represent evolving agent team configurations over time. These two classes of models can be combined to improve recognition accuracy, particularly for behaviors that appear similar in static snapshots. We evaluate our modeling techniques on large urban maps and position traces of two-person human teams performing MOUT behaviors in a customized version of Unreal Tournament (a commercially available first-person shooter game).