Harnessing Natural Fluctuations: Analogue Computer for Efficient Socially Maximal Decision Making

Each individual handles many tasks of finding the most profitable option from a set of options that stochastically provide rewards. Our society comprises a collection of such individuals, and the society is expected to maximise the total rewards, while the individuals compete for common rewards. Such collective decision making is formulated as the `competitive multi-armed bandit problem (CBP)', requiring a huge computational cost. Herein, we demonstrate a prototype of an analog computer that efficiently solves CBPs by exploiting the physical dynamics of numerous fluids in coupled cylinders. This device enables the maximisation of the total rewards for the society without paying the conventionally required computational cost; this is because the fluids estimate the reward probabilities of the options for the exploitation of past knowledge and generate random fluctuations for the exploration of new knowledge. Our results suggest that to optimise the social rewards, the utilisation of fluid-derived natural fluctuations is more advantageous than applying artificial external fluctuations. Our analog computing scheme is expected to trigger further studies for harnessing the huge computational power of natural phenomena for resolving a wide variety of complex problems in modern information society.

[1]  Alan M. Turing,et al.  Computability and λ-definability , 1937, Journal of Symbolic Logic.

[2]  H. Vincent Poor,et al.  Cognitive Medium Access: Exploration, Exploitation, and Competition , 2007, IEEE Transactions on Mobile Computing.

[3]  Cristopher Moore,et al.  A complex legacy , 2011 .

[4]  Song-Ju Kim,et al.  Tug-of-War Model for Multi-armed Bandit Problem , 2010, UC.

[5]  Hai Jiang,et al.  Medium access in cognitive radio networks: A competitive multi-armed bandit framework , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[6]  Song-Ju Kim,et al.  Single-photon decision maker , 2015, Scientific Reports.

[7]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[8]  Motoichi Ohtsu,et al.  Decision making based on optical excitation transfer via near-field interactions between quantum dots , 2014 .

[9]  Bee-Chung Chen,et al.  Explore/Exploit Schemes for Web Content Optimization , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[10]  Motoichi Ohtsu,et al.  Decision Maker based on Nanoscale Photo-excitation Transfer , 2013, Scientific reports.

[11]  Song-Ju Kim,et al.  Tug-of-war model for the two-bandit problem: Nonlocally-correlated parallel exploration via resource conservation , 2010, Biosyst..

[12]  Song-Ju Kim,et al.  Efficient decision-making by volume-conserving physical object , 2014, ArXiv.

[13]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[14]  Song-Ju Kim,et al.  Decision Maker using Coupled Incompressible-Fluid Cylinders , 2015, ArXiv.

[15]  Donald W. Davies.  The Bombe a Remarkable Logic Machine , 1999, Cryptologia.

[16]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[17]  Tim Roughgarden,et al.  Selfish routing and the price of anarchy , 2005 .

[18]  Song-Ju Kim,et al.  Amoeba-inspired Tug-of-War algorithms for exploration-exploitation dilemma in extended Bandit Problem , 2014, Biosyst..

[19]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[20]  W. R. Thompson.  ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[21]  R. Agrawal.  Sample mean based index policies by O(log n) regret for the multi-armed bandit problem , 1995, Advances in Applied Probability.

[22]  Song-Ju Kim,et al.  Amoeba-inspired algorithm for cognitive medium access , 2014 .

[23]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.