Planning with Stochastic Resource Profiles : An Application to Human-Robot Cohabitation

It is important for robotic agents to be respectful of the intentions of the human members cohabiting an environment and account for conflicts on the shared resources in the environment, in order to be acceptable members of human-robot ecosystems. In this paper we look at how maintaining predictive models of the human cohabitors in the environment can be used to inform the planning process of the robotic agents. We introduce an Integer Programming based planner as a general formulation of the “human-aware” planning problem and show how the proposed formulation can be used to model different behaviors of the robotic agent, showcasing compromise, opportunism or negotiation. Finally, we show how the proposed approach scales with the different parameters involved, and provide empirical evaluations to illustrate the pros and cons associated with the proposed style of planning. In environments where multiple agents are working independently, but utilizing shared resources, it is important for these agents to maintain belief models of other agents so as to act intelligently and prevent conflicts. In cases where some of these agents are humans, as in assistive robots in household environments, these are required (rather than desired) capabilities of robots in order to be “socially acceptable” this has been studied extensively under the umbrella of “human-aware” planning, both in the context of path planning (Sisbot et al. 2007; Kuderer et al. 2012) and in task planning (Cirillo, Karlsson, and Saffiotti 2009; Koeckemann, Pecora, and Karlsson 2014; Cavallo et al. 2014; Tomic, Pecora, and Saffiotti 2014). Probabilistic plan recognition can play an important role in this regard, because by not committing to a plan, that pre-assumes a particular plan for the other agent, it might be possible to minimize suboptimal (in terms of redundant or conflicting actions performed during the execution phase) behavior of the autonomous agent. Here we look at possible ways to minimize such suboptimal behavior by ways of compromise, opportunism or negotiation. There has been previous work (Beaudry, Kabanza, and Michaud 2010; Cirillo, Karlsson, and Saffiotti 2010) on some of the modeling aspects of the Figure 1: Architecture diagram the robot has partial beliefs of the world, which it uses to predict and plan. problem, in terms of planning with uncertainty in resources and constraints. In this paper we provide a unified framework of achieving these behaviors of the autonomous agents, particularly in such scenarios of human robot cohabitation. The general framework of the problem addressed in this work is shown in Figure 1. The autonomous agent, or the robot, is acting (with independent goals) in an environment co-habited with other agents (humans), who are similarly self-interested. The robot has a model of the other agents acting independently in its environment. These models may be partial and hence the robot can only make uncertain predictions on how the world will evolve with time. However, the resources in the environment are limited and are likely to be constrained by the plans of the other agents. The robot thus needs to reason about the future states of the environment in order to make sure that its own plans do not produce conflicting states with respect to the plans of the other agents. With the involvement of humans, however, the problem is more skewed against the robot, because humans would expect a higher priority on their plans robots that produce plans that clash with those of the humans, without any explanation, would be considered incompatible for such an ecosystem. Thus the robot will be expected to follow plans that preserve the human plans, rather than follow a globally optimal plan for itself. This aspect makes the current setting distinct from normal human robot teaming scenarios and produces a numFigure 2: The running example a human commander and a robot involved in an USAR setting, with constrained resources (medkits). ber of its own interesting challenges. How does the robot model the humans’ behavior? How does it plan to avoid friction with the human plans? If it is possible to communicate, how does it plan to negotiate and refine plans? These are the questions that we seek to address in this work. Our approach models human beliefs and defines resource profiles as abstract representations of the plans predicted on the basis of these beliefs of the human agents. The robot updates its own beliefs about the world upon receiving every new observation from its environment, and passes on the resultant profiles onto its own planner as shown in Figure 1. For the planning module, we introduce an IP-based planner that minimizes the overlap between these resource profiles and those produced by the robot’s own plan in order to maintain least conflicts with the predicted human tasks in the future. 1 Planning with Resource Profiles We will now go into details about each of the modules shown in Figure 1. We will be using a similar setting as the one described in (Talamadupula et al. 2014) (shown in Figure 2) as the running example throughout this discussion. The setting involves a commander CommX and a robot in a USAR (Urban Search and Rescue) scenario. The shared resources here are the two medkits some of the plans the commander can execute will lock the use of and/or change the position of these medkits, so that from the set of probable plans of the commander we can extract a probability distribution over the usage (or even the position) of the medkit over time based on the fraction of plans that conform to these facts. These resource availability profiles provide a way for the agents to minimize conflicts with the other agents. Before going into details about the planner that achieves this, we will first look at how the agents are modeled and how these profiles are computed in the next section. 1.1 The Belief Modeling Component The notion of modeling beliefs introduced by the authors in (Talamadupula et al. 2014) is adopted in this work and described briefly here. Beliefs about state are defined in terms of predicates bel(α, φ), where α is an agent with belief φ = true. Goals are defined by predicates goal(α, φ), where agent α has a goal φ. The set of all beliefs that the robot ascribes to α together represents the perspective for the robot of α. This is obtained by a belief model Belα of agent α, defined as { φ | bel(α, φ) ∈ Belself }, where Belself are the first-order beliefs of the robot (e.g., bel(self, at(self, room1))). The set of goals ascribed to α is similarly described by {goal(α, φ)|goal(α, φ) ∈ Belself}. Next, we turn our attention to the domain model Dα of the agent α that is used in the planning process. Formally, a planning problem Π = 〈Dα, πα〉 consists of the domain model Dα and the problem instance πα. The domain model of α is defined as Dα = 〈Tα, Vα, Sα, Aα〉, where Tα is a set of object types; Vα is a set of variables that describe objects that belong to Tα; Sα is a set of named first-order logical predicates over the variables Vα that describe the state; and Aα is a set of operators available to the agent. The action models a ∈ Aα are represented as a = 〈N,C,P,E〉where N denotes the name of that action; C is the cost of that action; P is the list of pre-conditions that must hold for the action a to be applicable; and E = {eff(a), eff−(a)} is a list of predicates in Sα that indicates the effects of applying the action. The transition function δ(·) determines the next state after the application of action a in state s as δ(a, s) = (s \ eff−(a)) ∪ eff(a), s ⊆ SR. For this work, we assume that the action models available to an agent are completely known to all the other agents in the scenario; that is, we rule out the possibility of beliefs on the models of other agents. The belief model, in conjunction with beliefs about the goals / intentions of another agent, will allow the robot to instantiate a planning problem πα = 〈Oα, Iα,Gα〉, where Oα is a set of objects of type t ∈ Tα; Iα is the initial state of the world, and Gα is a set of goals, which are both sets of the predicates from Sα initialized with objects from Oα. First, the initial state Iα is populated by all of the robot’s initial beliefs about the agent α, i.e. Iα = {φ | bel(α, φ) ∈ Belrobot}. Similarly, the goal is set to Gα = {φ | goal(α, φ) ∈ Belrobot}. Finally, the set of objects Oα consists of all the objects that are mentioned in either the initial state, or the goal description: Oα = {o | o ∈ (φ | φ ∈ (Iα ∪Gα))}. This planning problem instance (though not directly used in the robot’s planning process) enables the goal recognition component to solve the compiled problem instances. 1.2 The Goal Recognition Component It is unlikely for the robot to be aware of the goals of other humans in its environment completely, but it can be proactive in updating its beliefs incrementally based on observations of what the other agents are doing. To accommodate this, the robot’s current belief of α’s goal, Gα, is extended to a hypothesis goal set Ψα. The computation of this goal set can be done using planning graph (Blum and Furst 1995) methods. In the worst case, Ψα corresponds to all possible goals in the final level of the converged planning graph. Having further (domain-dependent) knowledge (e.g. in our scenario, information that CommX is only interested in triagerelated goals) can prune some of these goals by removing the goal conditions that are not typed on the triage variable. At this point we refer to the work of Ramirez and Geffner who Figure 3: Different types of profiles corresponding to the two recognized plans. in (Ramrez and Geffner 2010) provided a technique to compile the problem of goal recognition into a classical planning problem. Given a sequence of observations θ, the probability distribution Θ over G ∈ Ψα is recomputed by using a Bayesian update P (G|θ) ∝ P (θ|G), where the prior is approximated by the function P (θ|G) = 1/(1 + e−β∆(G,θ