In order to study the soil nail optimization of foundation pit and obtain the global optimal solution, this paper establishes the predictive model of response surface by combining uniform test design(UD) and the support vector machine(SVM) technology. In order to avoid the disadvantages of large amount of calculation and low efficiency in heuristic algorithm, and to gain the optimal model of the explicitly analytical function in gradient algorithm, based on the predictive model of response surface under the constraint conditions, the foundation pit soil nailing is optimized through the application of enumerative algorithm. Through an engineering example, it is showed that is a time-saving method and don't need explicitly analytical model. Meanwhile, it can obtain the global optimal solution.