On the Structure and Computation of Random Walk Times in Finite Graphs

We consider random walks in which the walk originates in one set of nodes and then continues until it reaches one or more nodes in a target set. The time required for the walk to reach the target set is of interest in understanding the convergence of Markov processes, as well as applications in control, machine learning, and social sciences. In this paper, we investigate the computational structure of the random walk times as a function of the set of target nodes, and find that the commute, hitting, and cover times all exhibit submodular structure, even in nonstationary random walks. We provide a unifying proof of this structure by considering each of these times as special cases of stopping times. We generalize our framework to Markov decision processes, in which the target sets and control policies are jointly chosen to minimize the travel times, leading to polynomial-time approximation algorithms for choosing target sets. Our results are validated through numerical study.

[1]  David J. Aldous,et al.  Lower bounds for covering times for reversible Markov chains and random walks on graphs , 1989 .

[2]  R. Srikant,et al.  Opinion dynamics in social networks: A local interaction game with stubborn agents , 2012, 2013 American Control Conference.

[3]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[4]  Peter Winkler,et al.  Maximum itting Time for Random Walks on Graphs , 1990, Random Struct. Algorithms.

[5]  Christel Baier,et al.  Principles of model checking , 2008 .

[6]  Miklós Simonovits,et al.  Random Walks in a Convex Body and an Improved Volume Algorithm , 1993, Random Struct. Algorithms.

[7]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[8]  Fern Y. Hunt.  An Algorithm for Identifying Optimal Spreaders in a Random Walk Model of Network Communication , 2016 .

[9]  Ufuk Topcu,et al.  Optimal temporal logic planning in probabilistic semantic maps , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Uriel Feige,et al.  A Tight Upper Bound on the Cover Time for Random Walks on Graphs , 1995, Random Struct. Algorithms.

[11]  Chen Avin,et al.  Efficient and robust query processing in dynamic environments using random walk techniques , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[12]  C. Banderier.  A Generalized Cover Time for Random Walks on Graphs , 2000 .

[13]  Calin Belta,et al.  MDP optimal control under temporal logic constraints , 2011, IEEE Conference on Decision and Control and European Control Conference.

[14]  François Fouss,et al.  Graph Nodes Clustering Based on the Commute-Time Kernel , 2007, PAKDD.

[15]  Nacim Ramdani,et al.  A Comprehensive Method for Reachability Analysis of Uncertain Nonlinear Hybrid Systems , 2016, IEEE Transactions on Automatic Control.

[16]  François Fouss,et al.  Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation , 2007, IEEE Transactions on Knowledge and Data Engineering.

[17]  Elizabeth L. Wilmer,et al.  Markov Chains and Mixing Times , 2008 .

[18]  Basel Alomair,et al.  Minimizing Convergence Error in Multi-Agent Systems Via Leader Selection: A Supermodular Optimization Approach , 2013, IEEE Transactions on Automatic Control.

[19]  Prabhakar Raghavan,et al.  The electrical resistance of a graph captures its commute and cover times , 2005, computational complexity.

[20]  Luh Yen,et al.  A family of dissimilarity measures between nodes generalizing both the shortest-path and the commute-time distances , 2008, KDD.

[21]  F. Göbel,et al.  Random walks on graphs , 1974 .

[22]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[23]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[24]  Uriel Feige,et al.  A Tight Lower Bound on the Cover Time for Random Walks on Graphs , 1995, Random Struct. Algorithms.

[25]  Radha Poovendran,et al.  A Supermodular Optimization Framework for Leader Selection Under Link Noise in Linear Multi-Agent Systems , 2012, IEEE Transactions on Automatic Control.

[26]  Calin Belta,et al.  Formal synthesis of control strategies for dynamical systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[27]  M. Karonski.  Collisions among Random Walks on a Graph , 1993 .

[28]  Purnamrita Sarkar,et al.  Random Walks in Social Networks and their Applications: A Survey , 2011, Social Network Data Analytics.

[29]  J. Delvenne,et al.  Random walks on graphs , 2004 .

[30]  G. Lawler.  Random Walk and the Heat Equation , 2010 .

[31]  Peter G. Doyle,et al.  Random Walks and Electric Networks: REFERENCES , 1987 .

[32]  Laurence A. Wolsey,et al.  An analysis of the greedy algorithm for the submodular set covering problem , 1982, Comb..

[33]  Edwin R. Hancock,et al.  Clustering and Embedding Using Commute Times , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Calin Belta,et al.  Optimal Control of Markov Decision Processes With Linear Temporal Logic Constraints , 2014, IEEE Transactions on Automatic Control.