Optimality of adaption based Mean Field control laws in leader-follower stochastic collective dynamics

We study the optimality properties of maximum likelihood ratio estimation based Mean Field (Nash Certainty Equivalence) control laws in a leader-follower stochastic collective dynamics model. In this formulation the leaders track a convex combination of their centroid together with a certain reference trajectory which is unknown to the followers, and each follower reacts by tracking the centroid of the leaders. The followers use a maximum likelihood estimator (based on a fixed ratio sample of the population of the leaders' trajectories) to identify the member of a given finite class of models which is generating the reference trajectory of the leaders. Subject to reasonable conditions, it is shown that each adaptive follower identifies the true reference trajectory model in finite time with probability one as the leaders' population goes to infinity. It is also shown that the leaders' control laws possess an almost sure ε-Nash equilibrium property with respect to all other leaders. In this paper we show that the system performance for the adaptive followers is almost surely ε-optimal with respect to the leaders.

[1]  Peter E. Caines,et al.  The NCE (Mean Field) Principle With Locality Dependent Cost Interactions , 2010, IEEE Transactions on Automatic Control.

[2]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[3]  P. Caines,et al.  Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[4]  Peter E. Caines,et al.  Mean Field (NCE) Formulation of Estimation Based Leader-Follower Collective Dynamics , 2011, Int. J. Robotics Autom..

[5]  Jean-Jacques E. Slotine,et al.  A theoretical study of different leader roles in networks , 2006, IEEE Transactions on Automatic Control.

[6]  P. E. Caines,et al.  A Note on the Consistency of Maximum Likelihood Estimates for Finite Families of Stochastic Processes , 1975 .

[7]  D. Helbing,et al.  Leadership, consensus decision making and collective behaviour in humans , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[8]  Dongbing Gu,et al.  Leader–Follower Flocking: Algorithms and Experiments , 2009, IEEE Transactions on Control Systems Technology.

[9]  Pedro Elosegui,et al.  Extension of the Cucker-Smale Control Law to Space Flight Formations , 2009 .

[10]  I. Couzin,et al.  Effective leadership and decision-making in animal groups on the move , 2005, Nature.

[11]  Tao Li,et al.  Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems , 2008, IEEE Transactions on Automatic Control.

[12]  J. Cruz,et al.  On the Stackelberg strategy in nonzero-sum games , 1973 .

[13]  Benjamin Van Roy,et al.  Markov Perfect Industry Dynamics with Many Firms , 2005 .

[14]  Minyi Huang,et al.  Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria , 2007, IEEE Transactions on Automatic Control.

[15]  Val E. Lambson.  Self-enforcing collusion in large dynamic markets , 1984 .

[16]  Tyrone E. Duncan,et al.  Evaluation of Likelihood Functions , 1968, Inf. Control..

[17]  Minyi Huang,et al.  Stochastic power control in wireless communication systems: analysis, approximate control algorithms and state aggregation , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).