Unsupervised Resource Allocation with Graph Neural Networks

We present an approach for maximizing a global utility function by learning how to allocate resources in an unsupervised way. We expect interactions between allocation targets to be important and therefore propose to learn the reward structure for near-optimal allocation policies with a GNN. By relaxing the resource constraint, we can employ gradient-based optimization in contrast to more standard evolutionary algorithms. Our algorithm is motivated by a problem in modern astronomy, where one needs to select—based on limited initial information—among 10 galaxies those whose detailed measurement will lead to optimal inference of the composition of the universe. Our technique presents a way of flexibly learning an allocation strategy by only requiring forward simulators for the physics of interest and the measurement process. We anticipate that our technique will also find applications in a range of allocation problems from social science studies to customer satisfaction surveys and exploration strategies of autonomous agents.

[1]  Georg Martius,et al.  Differentiation of Blackbox Combinatorial Solvers , 2020, ICLR.

[2]  P. Coles.  Cosmology: A Very Short Introduction , 2001 .

[3]  G. Dantzig.  Discrete-Variable Extremum Problems , 1957 .

[4]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[5]  Sanja Fidler,et al.  NerveNet: Learning Structured Policy with Graph Neural Networks , 2018, ICLR.

[6]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning Based Resource Allocation for V2V Communications , 2018, IEEE Transactions on Vehicular Technology.

[7]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[8]  Keke Gai,et al.  Optimal resource allocation using reinforcement learning for IoT content-centric services , 2018, Appl. Soft Comput..

[9]  Rui Xu,et al.  Discovering Symbolic Models from Deep Learning with Inductive Biases , 2020, NeurIPS.

[10]  Alejandro Ribeiro,et al.  Optimal Wireless Resource Allocation With Random Edge Graph Neural Networks , 2019, IEEE Transactions on Signal Processing.

[11]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[12]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[13]  Xia Hu,et al.  Policy-GNN: Aggregation Optimization for Graph Neural Networks , 2020, KDD.

[14]  Bernhard Schölkopf,et al.  On the design of consequential ranking algorithms , 2020, UAI.

[15]  Qinru Qiu,et al.  A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[16]  Robert J. Vanderbei,et al.  A Framework for Telescope Schedulers: With Applications to the Large Synoptic Survey Telescope , 2018, The Astronomical Journal.

[17]  Kristina Lerman,et al.  Resource allocation in the grid using reinforcement learning , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[18]  Chris Bebek,et al.  The Dark Energy Spectroscopic Instrument (DESI) , 2019, 1907.10688.

[19]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[20]  Zhan Gao,et al.  Resource Allocation via Graph Neural Networks in Free Space Optical Fronthaul Networks , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[21]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[22]  Shirley Ho,et al.  Learning Symbolic Physics with Graph Networks , 2019, ArXiv.

[23]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.