Pursuit of the Universal

Approaches to the verification of multi-agent systems are typically based on games or transition systems defined in terms of states and actions. However such approaches often ignore a key aspect of multiagent systems, namely that the agents’ actions require (and sometimes produce) resources. We briefly survey previous work on the verification of multi-agent systems that takes resources into account, and outline some key challenges for future work. 1 Verifying Autonomous Systems A multi-agent system (MAS) is a system that is composed of multiple interacting agents. An agent is an autonomous entity that has the ability to collect information, reason about it, and perform actions based on it in pursuit of its own goals or on behalf of others. Examples of agents are controllers for satellites, non-driver transport systems such as UAVs, health care systems, and even nodes in sensor networks. Multi-agent systems are ubiquitous. Many distributed software and hardware systems can be naturally modelled as multi-agent systems. Such systems are by the nature of their components extremely complex, and the interaction between components and their environment can lead to undesired behaviours that are difficult to predict in advance. With the increasing use of autonomous agents in safety critical systems, there is a growing need to verify that their behaviour conforms to the desired system specification, and over the last decade verification of multi-agent systems has become a thriving research area [24]. A key approach to the verification of MAS is model checking. Model checking involves checking whether a model of the system satisfies a temporal logic formula corresponding to some aspect of the system specification. Model checking has the advantage that it is a fully automated technique, which facilitates its use in the MAS development process. A wide range of approaches to model-checking MAS have been proposed in the literature, ranging from the adaptation of standard model-checking tools, e.g., [12,13] to the development of special-purpose model checkers for multi-agent systems, e.g., [22,27]. 1 Another strand of work focusses on theorem proving, e.g., [28], but such approaches typically require user interaction to guide the search for a proof. c © Springer International Publishing Switzerland 2016 A. Beckmann et al. (Eds.): CiE 2016, LNCS 9709, pp. 3–12, 2016. DOI: 10.1007/978-3-319-40189-8 1 4 N. Alechina and B. Logan 2 Resource-Bounded Agents In many multi-agent systems, agents are resource-bounded, in the sense that they require resources in order to act. Actions require time to complete and typically require additional resources depending on the application domain, for example energy or money. For many applications, the availability or otherwise of resources is critical to the properties we want to verify: a multi-agent system will have very different behaviours depending on the resource endowment of the agents that comprise it. For example, an agent with insufficient energy may be unable to complete a task in the time assumed by a team plan, if it has to recharge its battery before performing the task. However, with a few exceptions which we discuss below, previous work on verification of MAS abstracts away from the fact that many multi-agent systems consist of agents that need resources to operate and that those resources are limited. In particular, current state-of-the-art verification techniques and tools for MAS are unable to verify system properties that depend on the resource production and consumption of the agents comprising the MAS. In this paper we survey recent work in the emerging field of verification of resource-bounded agents, and highlight a number of challenges that must be overcome to allow practical verification of resource-bounded MAS. We argue that recent work on the complexity of model-checking for logics of strategic ability with resources offers the possibility of significant progress in the field, new verification approaches and tools, and the ability to verify the properties of a large, important class of autonomous system that were previously out of reach. 3 Model-Checking with Resources In this section we give a brief introduction to model-checking multi-agent systems and explain how standard model checking approaches can be extended with resources. In a model-checking approach to the verification of multi-agent systems, a MAS is represented by a finite state transition system. A state transition system consists of a set of states and transitions between them. Intuitively, each state of a MAS corresponds to a tuple of states of the agents and of the environment, and each transition corresponds to actions performed by the agents. Each state is labelled with atomic propositions that are true in that state. A standard assumption is that each state in the system has at least one outgoing transition (if a state is a deadlock state in the original MAS, we can model this by adding a transition to itself by some null action and labelling it with a ‘deadlock’ proposition). Properties of the system to be verified are expressed in an appropriate temporal logic L. The model-checking problem for L is, given a state transition system M (and possibly a state s) and an L formula φ, check whether φ is true in M (at state s). 2 There is work on model-checking infinite state transition systems, see, for example, [11], but in this paper we concentrate on the finite case. Verifying Systems of Resource-Bounded Agents 5 For multi-agent systems, a temporal logic of particular interest is Alternating Time Temporal Logic (ATL) [9]. ATL generalises other temporal logics such as Computation Tree Logic (CTL) [19] (which can be seen as a one-agent ATL) by introducing notions of strategic ability. ATL is interpreted over concurrent game structures (transition systems where edges correspond to a tuple of actions performed simultaneously by all the agents, see the example below). The language of ATL contains atomic propositions, boolean connectives ¬,∧, etc. and modalities 〈〈A〉〉©, 〈〈A〉〉 and 〈〈A〉〉U for each subset (or coalition in ATL terms) A of the set of all agents, which express the strategic ability of the coalition A. 〈〈A〉〉©φ means that the coalition of agents A has a choice of actions such that, regardless of what the other agents in the system do, φ will hold in the next state. 〈〈A〉〉 φ means that coalition A has a strategy to keep φ true forever, regardless of what the other agents do. A strategy is a choice of actions which either only depends on the current state (memoryless strategy) or on the finite history of the current state (perfect recall strategy). Finally, 〈〈A〉〉φU ψ means that A has a strategy to ensure that after finitely many steps ψ holds, and in all the states before that, φ holds. The model-checking problem for ATL can be solved in time polynomial in the size of the transition system and the property [9], and there exist model-checking tools for ATL, for example, MOCHA [10] and MCMAS [27]. Example. Fig. 1 illustrates a simple ATL model of a system with two agents, 1 and 2, and actions α, β, γ and idle. Action tuples on the edges show the actions of each agent, for example, in the transition from state sI to s, agent 1 performs action α and agent 2 performs idle. In this system, in state sI , agent 1 has a (memoryless) strategy to enforce that p holds eventually in the future no matter what agent 2 does, which can be expressed in ATL as 〈〈{1}〉〉 U p. Similarly, in sI agent 1 has a memoryless strategy to keep ¬p true forever, so 〈〈{1}〉〉 ¬p holds in sI . 3.1 Adding Resources In order to model multi-agent systems where agents’ actions produce and consume resources, it is necessary to modify the approach above in two ways. One is to add resource annotations to the actions in the transition system: for each individual action and each resource type, we need to specify how many units of this resource type the action produces or consumes. For example, suppose that there are two resource types, r1 and r2 (e.g., energy and money). Then we can specify that action α in Fig. 1 produces two units of r1 and consumes one unit of r2, action β consumes one unit of r1 and produces one unit of r2, action γ consumes five units of r1, and action idle does not produce or consume any resources. The second modification is to extend the temporal logic so that we can express properties related to resources. For example, we may want to express a property that a group of agents A can eventually reach a state satisfying φ or can maintain the truth of ψ forever, provided that they have available n1 units 6 N. Alechina and B. Logan of resource type r1 and n2 units of resource type r2. Such statements about coalitional ability under resource bounds can be expressed in an extension of ATL where coalitional modalities are annotated with a resource bound on the strategies available to the coalition. We call logics where every action is associated with produced and consumed resources and the syntax reflects resource requirements on agents, resource logics. To illustrate the properties resource logics allow us to express, consider the model in Fig. 1 with the production and consumption of resources by actions specified above. In this setting, we can verify if agent 1 can eventually enforce p provided that it has one unit of r2 in state sI , or whether the coalition of agents {1, 2} can achieve p under this resource bound by working together. There are surprisingly many different ways of measuring costs of strategies and deciding which actions are executable by the agents given the resources available to them, but under at least one possible semantics, the answer to the first question is no and to the second one yes, but the latter requires a perfect recall strategy (the two agents should loop between states sI and s until they produce a sufficient amount of resource r1, and then execute actions corresponding to the 〈γ, idle〉 transition