On the Emergence of Shortest Paths by Reinforced Random Walks

The co-evolution between network structure and functional performance is a fundamental and challenging problem whose complexity emerges from the intrinsic interdependent nature of structure and function. Within this context, we investigate the interplay between the efficiency of network navigation (i.e., path lengths) and network structure (i.e., edge weights). We propose a simple and tractable model based on iterative biased random walks where edge weights increase over time as function of the traversed path length. Under mild assumptions, we prove that biased random walks will eventually only traverse shortest paths in their journey towards the destination. We further characterize the transient regime proving that the probability to traverse non-shortest paths decays according to a power-law. We also highlight various properties in this dynamic, such as the trade-off between exploration and convergence, and preservation of initial network plasticity. We believe the proposed model and results can be of interest to various domains where biased random walks and de-centralized navigation have been applied.

[1]  Kenji Leibnitz,et al.  Biologically inspired self-adaptive multi-path routing in overlay networks , 2006, Commun. ACM.

[2]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[3]  Marco Dorigo,et al.  Ant colony optimization theory: A survey , 2005, Theor. Comput. Sci..

[4]  François Fouss,et al.  Randomized Shortest-Path Problems: Two Related Models , 2009, Neural Computation.

[5]  John H. Hartman,et al.  Data-Centric Routing in Sensor Networks using Biased Walk , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[6]  Andrea Passarella,et al.  A survey on content-centric technologies for the current Internet: CDN and P2P solutions , 2012, Comput. Commun..

[7]  Edward A. Codling,et al.  Random walk models in biology , 2008, Journal of The Royal Society Interface.

[8]  Paul R Moorcroft,et al.  Stochastic modelling of animal movement , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[9]  John N. Tsitsiklis,et al.  On the Convergence of Optimistic Policy Iteration , 2002, J. Mach. Learn. Res..

[10]  B. Davis,et al.  Reinforced random walk , 1990 .

[11]  Svante Janson,et al.  Limit theorems for triangular urn schemes , 2006 .

[12]  Svante Janson,et al.  Functional limit theorems for multitype branching processes and generalized Pólya urns , 2004 .

[13]  A survey of random processes with reinforcement , 2007, math/0610076.

[14]  Hosam M. Mahmoud,et al.  Polya Urn Models , 2008 .

[15]  Stefan Rotter,et al.  Emergence of Functional Specificity in Balanced Networks with Synaptic Plasticity , 2015, PLoS Comput. Biol..

[16]  Andrew R. McKinstry-Wu,et al.  Connectome: How the Brain’s Wiring Makes Us Who We Are , 2013 .

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Thomas L. Griffiths,et al.  Human memory search as a random walk in a semantic network , 2012, NIPS.

[19]  Svante Janson Functional limit theorems for multitype branching processes , 2004 .

[20]  Gade Krishna,et al.  A scalable peer-to-peer lookup protocol for Internet applications , 2012 .

[21]  E. D’Angelo The human brain project. , 2012, Functional neurology.

[22]  R. Durrett Probability: Theory and Examples , 1993 .

[23]  Svante Janson,et al.  Functional limit theorems for multitype branching processes and generalized P& olya urns , 2004 .

[24]  D. Bernstein,et al.  Some explicit formulas for the matrix exponential , 1993, IEEE Trans. Autom. Control..

[25]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.