Tuning hybrid distributed storage system digital twins by reinforcement learning

In this paper, we consider the problem of fine-tuning a discrete event simulator of distributed storage system by a neural network trained with reinforcement learning algorithms on real data. The simulator has a set of control parameters that affect its behaviour and can be tuned during the simulation. Variation of these parameters influences how realistic the simulation is. The problem of simulator tuning is equivalent to the discovery of an optimal control strategy that leads to sensible results. We investigate different optimization metrics and demonstrate the viability of the approach.

[1]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[2]  Jerry Hamann,et al.  Large Fabric Storage Area Networks: Fabric Simulator Development and Preliminary Performance Analysis , 2010 .

[3]  Federico Silla,et al.  Modeling and simulation of storage area networks , 2000, Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728).

[4]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[5]  Jairo R. Montoya-Torres,et al.  Simulation-optimization using a reinforcement learning approach , 2008, 2008 Winter Simulation Conference.

[6]  Feng Zhou,et al.  Simulation of fibre channel storage area network using SANSim , 2003, The 11th IEEE International Conference on Networks, 2003. ICON2003..

[7]  George F. Riley,et al.  The ns-3 Network Simulator , 2010, Modeling and Tools for Network Simulation.

[8]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[9]  J.S. Carson,et al.  Model verification and validation , 2002, Proceedings of the Winter Simulation Conference.

[10]  András Varga,et al.  An overview of the OMNeT++ simulation environment , 2008, SimuTools.

[11]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[12]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[13]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[14]  Federico Silla,et al.  A tool for the design and evaluation of fibre channel storage area networks , 2001, Proceedings. 34th Annual Simulation Symposium.

[15]  Fei Tao,et al.  Digital twin-driven product design, manufacturing and service with big data , 2017, The International Journal of Advanced Manufacturing Technology.

[16]  J. J. Serrano,et al.  Improving the execution of groups of simulations on a cluster of workstations and its application to storage area networks , 2001, Proceedings. 34th Annual Simulation Symposium.

[17]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.