Action selection in growing state spaces: Control of Network Structure Growth

The dynamical processes taking place on a network depend on its topology. Influencing the growth process of a network therefore has important implications on such dynamical processes. We formulate the problem of influencing the growth of a network as a stochastic optimal control problem in which a structural cost function penalizes undesired topologies. We approximate this control problem with a restricted class of control problems that can be solved using probabilistic inference methods. To deal with the increasing problem dimensionality, we introduce an adaptive importance sampling method for approximating the optimal control. We illustrate this methodology in the context of formation of information cascades, considering the task of influencing the structure of a growing conversation thread, as in Internet forums. Using a realistic model of growing trees, we show that our approach can yield conversation threads with better structural properties than the ones observed without control.

[1]  Shuang Li,et al.  COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution , 2015, NIPS.

[2]  Ran Dai,et al.  Optimal topology design for dynamic networks , 2011, IEEE Conference on Decision and Control and European Control Conference.

[3]  David K. Smith,et al.  Dynamic Programming and Optimal Control. Volume 1 , 1996 .

[4]  Duncan J. Watts,et al.  The Structural Virality of Online Diffusion , 2015, Manag. Sci..

[5]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[6]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[7]  Alessandro Vespignani,et al.  Epidemic spreading in scale-free networks. , 2000, Physical review letters.

[8]  Matthew O. Jackson,et al.  The Formation of Networks with Transfers Among Players , 2004, J. Econ. Theory.

[9]  S. P. Cornelius,et al.  Realistic control of network dynamics , 2013, Nature Communications.

[10]  Vicenç Gómez,et al.  A likelihood-based framework for the analysis of discussion threads , 2012, World Wide Web.

[11]  Prasanna Gai,et al.  Contagion in financial networks , 2010, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[12]  Yana Volkovich,et al.  When the Wikipedians Talk: Network and Tree Structure of Wikipedia Discussion Pages , 2011, ICWSM.

[13]  Daniel Polani,et al.  Information Theory of Decisions and Actions , 2011 .

[14]  Christos Faloutsos,et al.  Patterns of Cascading Behavior in Large Blog Graphs , 2007, SDM.

[15]  Víctor M Eguíluz,et al.  Epidemic threshold in structured scale-free networks. , 2002, Physical review letters.

[16]  Evangelos Theodorou,et al.  Relative entropy and free energy dualities: Connections to Path Integral and KL control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[17]  Albert-László Barabási,et al.  Target control of complex networks , 2014, Nature Communications.

[18]  B. Mohar,et al.  How to compute the Wiener index of a graph , 1988 .

[19]  J. Slotine,et al.  Spectrum of controlling and observing complex networks , 2015, Nature Physics.

[20]  Matthew O. Jackson,et al.  The Evolution of Social and Economic Networks , 2002, J. Econ. Theory.

[21]  Le Song,et al.  Shaping Social Activity by Incentivizing Users , 2014, NIPS.

[22]  Eric Moulines,et al.  Comparison of resampling schemes for particle filtering , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[23]  Albert-László Barabási,et al.  Controllability of complex networks , 2011, Nature.

[24]  Damon Centola,et al.  The Spread of Behavior in an Online Social Network Experiment , 2010, Science.

[25]  Jon Kleinberg,et al.  Maximizing the spread of influence through a social network , 2003, KDD '03.

[26]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[27]  Paolo Giudici,et al.  Graphical Network Models for International Financial Flows , 2016 .

[28]  Rafael E. Banchs,et al.  The structure of political discussion networks: a model for the analysis of online deliberation , 2010, J. Inf. Technol..

[29]  H. Kappen.  Linear theory for control of nonlinear stochastic systems. , 2004, Physical review letters.

[30]  Éva Tardos,et al.  Maximizing the Spread of Influence through a Social Network , 2015, Theory Comput..

[31]  Vicenç Gómez,et al.  Optimal control as a graphical model inference problem , 2009, Machine Learning.

[32]  Vicenç Gómez,et al.  Policy Search for Path Integral Control , 2014, ECML/PKDD.

[33]  Songyang Lao,et al.  Enhancing complex network controllability by minimum link direction reversal , 2015 .

[34]  Marc Toussaint,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2012, Robotics: Science and Systems.

[35]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[36]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[37]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[38]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[39]  Andrea Baronchelli,et al.  The spontaneous emergence of conventions: An experimental study of cultural evolution , 2015, Proceedings of the National Academy of Sciences.

[40]  Vicenç Gómez,et al.  Statistical analysis of the social network and discussion threads in slashdot , 2008, WWW.

[41]  Fredrik Gustafsson,et al.  On Resampling Algorithms for Particle Filters , 2006, 2006 IEEE Nonlinear Statistical Signal Processing Workshop.

[42]  Hilbert J. Kappen,et al.  Adaptive Importance Sampling for Control and Inference , 2015, ArXiv.

[43]  Rama Cont,et al.  RESILIENCE TO CONTAGION IN FINANCIAL NETWORKS , 2010, 1112.5687.

[44]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[45]  Xinghuo Yu,et al.  Effective Augmentation of Complex Networks , 2016, Scientific Reports.

[46]  Y. Lai,et al.  Optimizing controllability of complex networks by minimum structural perturbations. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.