Is Selection Optimal for Scale-Free Small Worlds?

The ‘no free lunch theorem’ claims that for the set of all problems no algorithm performs better than random search and, thus, selection can be advantageous only on a limited set of problems. In this paper we investigate how the topological structure of the environment influences algorithmic efficiency. We study the performance of algorithms, using selective learning, reinforcement learning, and their combinations, in random, scale-free, and scale-free small world (SFSW) environments. The learning problem is to search for novel, not-yet-found information. We ran our experiments on a large news site and on its downloaded portion. Controlled experiments were performed on this downloaded portion: we modified the topology, but preserved the publication time of the news. Our empirical results show that the selective learning is the most efficient in SFSW topology. In non-small world topologies, however, the combination of the selective and reinforcement learning algorithms performs the best.

[1]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[2]  Filippo Menczer,et al.  Complementing search engines with online web mining agents , 2003, Decis. Support Syst..

[3]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[4]  G. K. Smelser.  The structure of the eye , 1961 .

[5]  Maja J. Mataric,et al.  Reinforcement Learning in the Multi-Robot Domain , 1997, Auton. Robots.

[6]  Andreas Stafylopatis,et al.  Autonomous vehicle navigation using evolutionary reinforcement learning , 1998, Eur. J. Oper. Res..

[7]  G. Kampis.  Self-modifying systems in biology and cognitive science , 1991 .

[8]  Thorsten Joachims,et al.  A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization , 1997, ICML.

[9]  András Lörincz,et al.  Fast adapting value estimation-based hybrid architecture for searching the world-wide web , 2002, Appl. Soft Comput..

[10]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[11]  András Lörincz,et al.  Erratum , 2007, Neural Computation.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Knut Magne Risvik,et al.  Search engines and Web dynamics , 2002, Comput. Networks.

[14]  C. Clark,et al.  Dynamic State Variable Models in Ecology: Methods and Applications , 2001 .

[15]  Jenny Edwards,et al.  An adaptive model for optimizing performance of an incremental web crawler , 2001, WWW '01.

[16]  A. Barabasi,et al.  Scale-free characteristics of random networks: the topology of the world-wide web , 2000 .

[17]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[18]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[19]  Mauro Annunziato,et al.  Artificial Life Optimization over Complex Networks , 2003 .

[20]  András Lörincz,et al.  Adaptive Highlighting Of Links To Assist Surfing On The Internet , 2005, Int. J. Inf. Technol. Decis. Mak..

[21]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[22]  András Lörincz,et al.  Intelligent High-Performance Crawlers Used to Reveal Topic-Specific Structure of WWW , 2002, Int. J. Found. Comput. Sci..

[23]  Hector Garcia-Molina,et al.  Effective page refresh policies for Web crawlers , 2003, TODS.

[24]  Diego Calvanese,et al.  Unifying Class-Based Representation Formalisms , 2011, J. Artif. Intell. Res..

[25]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[26]  Stephen Jones,et al.  Mutualism Promotes Diversity and Stability in a Simple Artificial Ecosystem , 2002, Artificial Life.

[27]  Arnon Rungsawang,et al.  Learnable topic-specific web crawler , 2002, J. Netw. Comput. Appl..

[28]  Daniel Boley,et al.  Principal Direction Divisive Partitioning , 1998, Data Mining and Knowledge Discovery.

[29]  Bernard Manderick,et al.  Extended Replicator Dynamics as a Key to Reinforcement Learning in Multi-agent Systems , 2003, ECML.

[30]  Vilmos Csányi.  Evolutionary systems and society : a general theory of life, mind, and culture , 1989 .

[31]  John M. Fryxell,et al.  Individual Behavior and Community Dynamics , 1997, Population and Community Biology Series.

[32]  Toshiyuki Kondo,et al.  A Reinforcement Learning with Evolutionary State Recruitment Strategy for Autonomous Mobile Robots Control , 2003 .

[33]  Jon Kleinberg,et al.  The Structure of the Web , 2001, Science.

[34]  Gerald Tesauro,et al.  Temporal difference learning and TD-Gammon , 1995, CACM.