Increasing traffic and bandwidth requirements bring challenges to the next generation wireless networks (5G). As one of the main technology in 5G networks, Ultra-Dense Network (UDN) can be used to improve network coverage. In this paper, a radio over fiber based model is proposed to solve the load balancing problem in ultra-dense network. Stochastic differential game is introduced for the load balancing algorithm, and optimal load allocated to each access point (RAP) are formulated as Nash Equilibrium. It is proved that the optimal load can be achieved and the stochastic differential game based scheme is applicable and acceptable. Numerical results are given to prove the effectiveness of the optimal algorithm.