QMORA: A Q-Learning based Multi-objective Resource Allocation Scheme for NFV Orchestration

To satisfy the various quality-of-service (QoS) re-quirements with minimum network costs, network functions virtualization (NFV) is proposed as an emerging wireless architecture that migrates network functions from dedicated hardware appliances to software instances running in virtual computing platforms. One crucial issue in NFV is to solve the orchestration of virtualized network functions (VNFs) to reduce costs and to improve the management flexibility of telecommunications service providers (TSPs). Particular, multiple objectives are required to be considered for orchestrating VNFs in order to achieve overall system performance. This can be optimally solved in small scale using integer linear programming (ILP) algorithms with high accuracy but low time efficiency. On the other hand, heuristic algorithms can be applied for solving part of the objectives in NFV resource allocation with high time efficiency but low accuracy. To tackle the above challenges, QMORA, a ${Q}$-learning based multi-objective resource allocation approach, is proposed to solve multi-objective optimization in NFV orchestration (NFVO) efficiently and accurately. Particularly, the approach includes reinforcement learning module and VNFs placement module. Reinforcement learning module is responsible for generating the “best” candidate paths. VNFs placement module is responsible for selecting optimal nodes on the generated candidate paths to host VNFs required for flows. The simulation results in the real ISP topology show that the proposed QMORA can balance the multi-objective including maximizing number of flows admitted to the network, minimizing path stretch, balancing the load among VNF instances and minimizing link occupation rate compared with other heuristic approaches.