Asynchronous Decision Making for Decentralised Autonomous Systems

George M. Mathews Doctor of Philosophy The University of Sydney March 2008 Asynchronous Decision Making for Decentralised Autonomous Systems This thesis is concerned with the design of decentralised controllers for distributed autonomous systems. This problem is encountered in any system which consists of multiple interacting autonomous entities that are required to make local decisions while pursuing a common objective. This thesis has its foundations in the design of optimal controllers for general decentralised stochastic systems. To overcome the computational complexity of the optimal solutions, a scheme is proposed based on open loop feedback control. Under this scheme each controller maintains a synchronised belief about the state of the system and collaborates with the other controllers in developing an open loop plan. The planning problem is formulated as an optimisation problem over the set of all individual control plans. Classical solution methods to this problem require all the information to be communicated to a single location, where standard algorithms are used to compute the optimal set of plans. Although this may be appropriate for small systems, it becomes computationally infeasible for large system containing many controllers and decentralised methods must be used. The main contribution of this thesis is a decentralised and asynchronous algorithm for this optimisation problem. Attention is focused on continuous problems, where the expected cost-to-go is a smooth function of the individual plans of each controller and gradient-based methods can be used. The proposed asynchronous algorithm allows each controller to generate an initial plan and then incrementally refine it, while intermittently communicating these refinements to the other controllers in the system. This method explicitly takes into account the possible communication delays between controllers. A convergence analysis is performed and a condition derived that intuitively relates these delays and the inter-controller coupling structure to the rate at which each controller can refine their local plan. The coupling between two controllers is determined by the curvature of the objective function in the subspace containing their local plans. If the coupling is known to the controllers, it can be used to prioritise the use of the available communication bandwidth, with only the highly coupled controllers communicating. However, the coupling structure requires detailed information on the 2nd order derivatives of the objective function and is generally not feasible to compute. To overcome this, an approximation method is developed that allows each controller to estimate the coupling. This allows each controller to dynamically determine which other controllers it must communicate with. Finally, this scheme is applied to the control of multi-sensor systems undertaking search and tracking tasks. For these sensing systems it is demonstrated that the inter-controller coupling can be interpreted as the overlap in the information obtained by the future observations of each sensor. It is also shown that this often results in a sparse coupling structure, with each controller only required to communicate with a few other coupled controllers.