Foundations of Assisted Cognition Systems

reasoning. However, such an integration requires us to bridge the gap between continuous sensor data and discrete, high-level representations. We will develop techniques that can bridge this gap by clustering a user’s data into discrete activity segments, which are path segments during which a user shows highly predictable behavior. Such segments can be learned for both indoor and outdoor environments using expectation maximization (EM) along with location data and an annotated map of the environment. Another challenge in location estimation is the fact that users frequently switch between environments with vastly different structure and sensor coverage. For example, a location system has to be able to track a person in her home, on her way to the coffee shop, and during her walk through the park. We will develop adaptive filters that are able to switch among or use simultaneously different map representations and different representations of uncertainty using discrete grid filters, particle filters, and / or Kalman filters whenever appropriate. 2.2 Behavior, Goal, and Plan Recognition Behavior recognition explains physical motion in terms of sets of meaningful activities that extend over time. Goal recognition relates behaviors to the performer’s intention to bring about some state of affairs. Behaviors and goals can be hierarchically organized into plans. Representing and reasoning about plans and goals requires us to advance the state of the art in probabilistic dynamic models in two directions: • Representing hierarchically structured complex activities with relative and absolute metric temporal constraints between substeps leads us to develop new algorithms for hierarchical statistical representations. We are focusing in particular on extensions to the expressive hidden semi-Markov models (HHSMM) formalism. As an example, to track a person preparing breakfast, the system needs to reason about the relative order and duration of subactivities such as setting the table, preparing tea, toasting bread, etc. • Many of the entities in our domain are inherently relational. For example, in order to determine how much time a person spends socializing (an important measure of health in the elderly) we need to reason about relationships over sets of individuals. Actions that may be performed in different ways at different times are most naturally captured by slot/filler relationships between an instance of the action and its parameters. Even for handling locations it can be useful to explicitly represent and reason about the functional relationship between an object and a position in space (e.g., “the place I left my car, wherever that is”). Our work on relational Markov models (RMMs) and dynamic probabilistic relational models (DPRMs) will allow us to represent and reason about such entities in a scalable manner; see Fig. 2(c) and Sections 3.3 and 4.3 The combination of hierarchical and relational probabilistic models provides a language and basic inference algorithms for probabilistic plan recognition. Fundamental research challenges remain, however, in defining the content of those models. In particular, Assisted Cognition systems need to be able to recognize when the user has made an error in performing (or failing to perform) a task and may require assistance. 2.3 Modeling Cognitive Impairments One general strategy for error detection we are developing is based on online model selection, which can be used to detect surprising or unusual behaviors. Part of this approach may involve creating error models that target the particular kinds of cognitive errors the user population may make, based on an underlying theory of behavioral impairments resulting from specific neurological deficits. Such models support precise error detection and intervention strategies.