Differentially Private Convolutional Neural Networks with Adaptive Gradient Descent

Deep learning achieves remarkable success in the fields of target detection, computer vision, natural language processing, and speech recognition. However, traditional deep learning models may suffer the privacy risk due to some training data involve sensitive information, such as the medical histories, location information and face images. Attackers can exploit the implicit information to recover the sensitive information from the training data. In order to protecting privacy of deep learning model, we develop a novel optimization algorithm called DPAGDCNN for convolution neural network which cooperates differential privacy technique. Specifically, DPAGD-CNN allocates privacy budgets more carefully in each iteration, rather than assigning a fixed privacy budget per iteration. We theoretically prove that our approach can protect the privacy of training data and it achieves higher classification accuracy under the moderate privacy budget in the MNIST and CIFAR-10 datasets.