In this paper, we consider a wireless federated learning (FL) system concerning differential privacy (DP) guarantee, where multiple edge devices collaboratively train a shared model under the coordination of a central base station (BS) through over-the-air computation (AirComp). However, due to the heterogeneity of wireless links, it is difficult to achieve the optimal trade-off between model privacy and accuracy during the FL model aggregation. To address this issue, we propose to utilize the reconfigurable intelligent surface (RIS) technology to mitigate the communication bottleneck in FL by reconfiguring the wireless propagation environment. Specifically, we aim to minimize the model optimality gap while strictly meeting the DP and transmit power constraints. This is achieved by jointly optimizing the device transmit power, artificial noise, and phase shifts at RIS, followed by developing a two-step alternating minimization framework. Simulation results will demonstrate that the proposed RIS-assisted FL model achieves a better trade-off between accuracy and privacy than the benchmarks.