DeepViNE: Virtual Network Embedding with Deep Reinforcement Learning

Virtual Network Embedding (VNE) is a crucial problem in network virtualization. Prior work on VNE is mainly focused on optimization-based solutions that are carefully constructed and tuned under specific assumptions about resource demands brought by virtual networks. Recently, a few works have appeared on automating the design of VNE solutions that work well under general virtual resource demands using Deep Reinforcement Learning (DRL). These works, however, still rely on manual selection of relevant problem features required in the DRL approach. In this work, we develop a DRL-based VNE solution called DeepViNE, which automates the selection of problem features required in the DRL approach. The key idea is to encode physical and virtual networks as two-dimensional images, which are then perceivable by a convolutional deep neural network. To speed up learning and algorithm convergence, we also design a strategy to limit the number of actions required by the learning agent, while still allowing suitable exploration of the solution space. We evaluate the convergence and performance of DeepViNE using simulations, and compare it with several existing algorithms. The results show that DeepViNE learns an embedding policy that improves upon the performance of other simulated algorithms by at least 11%.