In a typical edge computing paradigm, multiple edge servers are located near the end users to provide augmented computation and bandwidth. As the resources of the edge servers are limited, precisely predicting the workload of different edge servers can be of great importance in efficiently utilizing the edges. However, due to the dynamics of end users, the workload in the edge servers have lots of spikes. In this paper, we consider a deep learning method to predict the resource usage of edge servers. The main idea is to use graph neural network (GNN) to capture the interconnected topology of the edge servers. The edge servers that are close in proximity often have similar resource load patterns. The GNN is then concatenated with a LSTM layer to output the prediction value. The algorithm is evaluated in a real edge computing data set. The results show that our algorithm have high accuracy in predicting the resource usage. In addition, it outperforms other SOTA algorithms significantly.