In this paper, we address the safety of data-driven control for contact-rich manipulation. We propose to restrict the controller’s action space to keep the system in a set of safe states. In the absence of an analytical model, we show how Gaussian Processes (GP) can be used to approximate safe sets. We disable inputs for which the predicted states are likely to be unsafe using the GP. Furthermore, we show how locally designed feedback controllers can be used to improve the execution precision in the presence of modelling errors. We demonstrate the benefits of our method on a pushing task with a variety of dynamics, by using known and unknown surfaces and different object loads. Our results illustrate that the proposed approach significantly improves the performance and safety of the baseline controller.