Memory Effect on Adaptive Decision Making with a Chaotic Semiconductor Laser

We investigate the effect of a memory parameter on the performance of adaptive decision making using a tug-of-war method with the chaotic oscillatory dynamics of a semiconductor laser. We experimentally generate chaotic temporal waveforms of the semiconductor laser with optical feedback and apply them for adaptive decision making in solving a multiarmed bandit problem that aims at maximizing the total reward from slot machines whose hit probabilities are dynamically switched. We examine the dependence of making correct decisions on different values of the memory parameter. The degree of adaptivity is found to be enhanced with a smaller memory parameter, whereas the degree of convergence to the correct decision is higher for a larger memory parameter. The relations among the adaptivity, environmental changes, and the difficulties of the problem are also discussed considering the requirement of past decisions. This examination of ultrafast adaptive decision making highlights the importance of memorizing past events and paves the way for future photonic intelligence.

[1]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[2]  Jun Muramatsu,et al.  Secure key distribution using correlated randomness in lasers driven by common random light. , 2012, Physical review letters.

[3]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[4]  Atsushi Uchida,et al.  Tb/s physical random bit generation with bandwidth-enhanced chaos in three-cascaded semiconductor lasers. , 2015, Optics express.

[5]  Song-Ju Kim,et al.  Tug-of-war model for the two-bandit problem: Nonlocally-correlated parallel exploration via resource conservation , 2010, Biosyst..

[6]  Song-Ju Kim,et al.  Efficient decision-making by volume-conserving physical object , 2014, ArXiv.

[7]  Mikio Hasegawa,et al.  Improving throughput using multi-armed bandit algorithm for wireless LANs , 2018 .

[8]  Atsushi Uchida,et al.  Laser dynamical reservoir computing with consistency: an approach of a chaos mask signal. , 2016, Optics express.

[9]  A. Uchida,et al.  Fast physical random bit generation with chaotic semiconductor lasers , 2008 .

[10]  Song-Ju Kim,et al.  Single-photon decision maker , 2015, Scientific Reports.

[11]  Yuta Terashima,et al.  Real-time fast physical random number generator with a photonic integrated circuit. , 2017, Optics express.

[12]  T. L. Lai Andherbertrobbins.  Asymptotically Efficient Adaptive Allocation Rules , 1985 .

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  T. Nakagaki,et al.  Intelligence: Maze-solving by an amoeboid organism , 2000, Nature.

[15]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[16]  Oliver Kroemer,et al.  Combining active learning and reactive control for robot grasping , 2010, Robotics Auton. Syst..

[17]  Song-Ju Kim,et al.  Performance evaluation of decision making using chaotic semiconductor laser based on tug-of-war theory , 2017 .

[18]  H. Robbins,et al.  Asymptotically efficient adaptive allocation rules , 1985 .

[19]  Song-Ju Kim,et al.  Ultrafast photonic reinforcement learning based on laser chaos , 2017, Scientific Reports.

[20]  Motoichi Ohtsu,et al.  Decision making based on optical excitation transfer via near-field interactions between quantum dots , 2014 .

[21]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[22]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[23]  Albert Y. Zomaya,et al.  Trends and challenges in cloud datacenters , 2014, IEEE Cloud Computing.

[24]  Jun Muramatsu,et al.  Information-theoretic secure key distribution based on common random-signal induced synchronization in unidirectionally-coupled cascades of semiconductor lasers. , 2013, Optics express.