A Compressive Privacy approach to Generalized Information Bottleneck and Privacy Funnel problems

Abstract This paper explores a Compressive Privacy (CP) methodology for optimal tradeoff between utility gain and privacy loss. CP represents a dimension-reduced subspace design of optimally desensitized query that may be safely shared with the public. Built upon the information and estimation theory, this paper proposes a “differential mutual information” (DMI) criterion to safeguard the privacy protection (PP). Algorithmically, DMI-optimal solutions can be derived via the Discriminant Component Analysis (DCA). Moreover, DCA has two machine learning variants (one in the original space and another is the kernel space) good for supervised learning applications. By extending the notion of DMI to the utility gain and privacy loss, CP unifies the conventional Information Bottleneck (IB) and Privacy Funnel (PF) and lead to two constrained optimizers, named Generalized Information Bottleneck (GIB) and Generalized Privacy Funnel (GPF). In the supervised learning environments, DCA can be further extended to a DUCA machine learning variant to reach an optimal tradeoff between utility gain and privacy loss. Finally, for fast convergence, a golden-section iterative method is developed particularly for solving the two constrained optimization problems: GIB and GPF.

[1]  Sun-Yuan Kung,et al.  Discriminant component analysis for privacy protection and visualization of big data , 2017, Multimedia Tools and Applications.

[2]  Flávio du Pin Calmon,et al.  Privacy against statistical inference , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[4]  Ali Makhdoumi,et al.  Privacy-utility tradeoff under statistical uncertainty , 2013, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Sun-Yuan Kung,et al.  A compressive multi-kernel method for privacy-preserving machine learning , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[6]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[7]  David G. Stork,et al.  Pattern Classification , 1973 .

[8]  S.Y. Kung,et al.  Compressive Privacy: From Information\/Estimation Theory to Machine Learning [Lecture Notes] , 2017, IEEE Signal Processing Magazine.

[9]  Mehryar Mohri,et al.  Two-Stage Learning Kernel Algorithms , 2010, ICML.

[10]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[11]  Moni Naor,et al.  Our Data, Ourselves: Privacy Via Distributed Noise Generation , 2006, EUROCRYPT.

[12]  Sun-Yuan Kung,et al.  Collaborative PCA/DCA Learning Methods for Compressive Privacy , 2017, ACM Trans. Embed. Comput. Syst..

[13]  S. Kung.  Kernel Methods and Machine Learning , 2014 .

[14]  Muriel Médard,et al.  From the Information Bottleneck to the Privacy Funnel , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[15]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[16]  Gal Chechik,et al.  Information Bottleneck for Gaussian Variables , 2003, J. Mach. Learn. Res..