Stochastic approximation methods-Powerful tools for simulation and optimization: A survey of some recent work on multi-agent systems and cyber-physical systems

Stochastic approximation methods have found extensive and diversified applications. Recent emergence of networked systems and cyber-physical systems has generated renewed interest in advancing stochastic approximation into a general framework to support algorithm development for information processing and decisions in such systems. This paper presents a survey on some recent developments in stochastic approximation methods and their applications. Using connected vehicles in platoon formation and coordination as a platform, we highlight some traditional and new methodologies of stochastic approximation algorithms and explain how they can be used to capture essential features in networked systems. Distinct features of networked systems with randomly switching topologies, dynamically evolving parameters, and unknown delays are presented, and control strategies are provided.

[1]  G. Yin,et al.  Stability of a pure random delay system with two-time-scale Markovian switching , 2012 .

[2]  G. Yin,et al.  Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications , 2004 .

[3]  Le Yi Wang,et al.  Asynchronous Stochastic Approximation Algorithms for Networked Systems: Regime-Switching Topologies and Multiscale Structure , 2013, Multiscale Model. Simul..

[4]  Le Yi Wang,et al.  Control of vehicle platoons for highway safety and efficient utility: Consensus with communications and vehicle dynamics , 2014, Journal of Systems Science and Complexity.

[5]  Richard S. Varga,et al.  Matrix Iterative Analysis , 2000, The Mathematical Gazette.

[6]  J.E. Mazo,et al.  Digital communications , 1985, Proceedings of the IEEE.

[7]  G. Yin,et al.  Hybrid Switching Diffusions: Properties and Applications , 2009 .

[8]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[9]  T. Moon.  Error Correction Coding: Mathematical Methods and Algorithms , 2005 .

[10]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[11]  G. Yin,et al.  Hybrid Switching Diffusions , 2010 .

[12]  Craig W. Reynolds.  Flocks, herds, and schools: a distributed behavioral model , 1987, SIGGRAPH.

[13]  Yong Zeng.  Discrete-time markov chains: two-time-scale methods and applications, G. George Yin and Qing Zhang, Springer, New York, U.S.A., 2005, 347 pp., ISBN 0-387-21948-X , 2006 .

[14]  Le Yi Wang,et al.  Stochastic Recursive Algorithms for Networked Systems with Delay and Random Switching: Multiscale Formulations and Asymptotic Properties , 2011, Multiscale Model. Simul..

[15]  J. Toner,et al.  Flocks, herds, and schools: A quantitative theory of flocking , 1998, cond-mat/9804180.

[16]  Le Yi Wang,et al.  Asymptotic properties of consensus-type algorithms for networked systems with regime-switching topologies , 2011, Autom..

[17]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[18]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[19]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[20]  Vicsek,et al.  Novel type of phase transition in a system of self-driven particles. , 1995, Physical review letters.

[21]  W ReynoldsCraig.  Flocks, herds and schools: A distributed behavioral model , 1987 .