Entropy-SGD optimizes the prior of a PAC-Bayes bound: Data-dependent PAC-Bayes priors via differential privacy
暂无分享,去创建一个
[1] Anand D. Sarwate,et al. Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..
[2] Ian Goodfellow,et al. Deep Learning with Differential Privacy , 2016, CCS.
[3] Cynthia Dwork,et al. Differential Privacy , 2006, ICALP.
[4] Nathan Srebro,et al. Exploring Generalization in Deep Learning , 2017, NIPS.
[5] Alexander J. Smola,et al. Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo , 2015, ICML.
[6] Ohad Shamir,et al. Global Non-convex Optimization with Discretized Diffusions , 2018, NeurIPS.
[7] Shiliang Sun,et al. PAC-bayes bounds with data dependent priors , 2012, J. Mach. Learn. Res..
[8] Matus Telgarsky,et al. Spectrally-normalized margin bounds for neural networks , 2017, NIPS.
[9] Kunal Talwar,et al. Mechanism Design via Differential Privacy , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).
[10] Geoffrey E. Hinton,et al. Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.
[11] Rebecca N. Wright,et al. Differential privacy: an exploration of the privacy-utility landscape , 2013 .
[12] Ryota Tomioka,et al. Norm-Based Capacity Control in Neural Networks , 2015, COLT.
[13] Toniann Pitassi,et al. Preserving Statistical Validity in Adaptive Data Analysis , 2014, STOC.
[14] Hiroshi Nakagawa,et al. Differential Privacy without Sensitivity , 2016, NIPS.
[15] François Laviolette,et al. PAC-Bayesian Bounds based on the Rényi Divergence , 2016, AISTATS.
[16] Matus Telgarsky,et al. Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis , 2017, COLT.
[17] Davide Anguita,et al. Differential privacy and generalization: Sharper bounds with applications , 2017, Pattern Recognit. Lett..
[18] Gintare Karolina Dziugaite,et al. Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data , 2017, UAI.
[19] H. Scheffé. A Useful Convergence Theorem for Probability Distributions , 1947 .
[20] Jürgen Schmidhuber,et al. Flat Minima , 1997, Neural Computation.
[21] David A. McAllester. PAC-Bayesian model averaging , 1999, COLT '99.
[22] David A. McAllester,et al. A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks , 2017, ICLR.
[23] Stefano Soatto,et al. Entropy-SGD: biasing gradient descent into wide valleys , 2016, ICLR.
[24] Stefano Soatto,et al. Emergence of invariance and disentangling in deep representations , 2017 .
[25] Tong Zhang,et al. Information-theoretic upper and lower bounds for statistical estimation , 2006, IEEE Transactions on Information Theory.
[26] Yee Whye Teh,et al. Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.
[27] Quoc V. Le,et al. A Bayesian Perspective on Generalization and Stochastic Gradient Descent , 2017, ICLR.
[28] Toniann Pitassi,et al. Generalization in Adaptive Data Analysis and Holdout Reuse , 2015, NIPS.
[29] Christos Dimitrakakis,et al. Robust and Private Bayesian Inference , 2013, ALT.
[30] Peter Grünwald,et al. A Tight Excess Risk Bound via a Unified PAC-Bayesian-Rademacher-Shtarkov-MDL Complexity , 2017, ALT.
[31] Ben London,et al. A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent , 2017, NIPS.
[32] Raef Bassily,et al. Algorithmic stability for adaptive data analysis , 2015, STOC.
[33] John Langford,et al. (Not) Bounding the True Error , 2001, NIPS.
[34] O. Kallenberg. Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.
[35] Jonathan C. Mattingly,et al. Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise , 2002 .
[36] Raef Bassily,et al. Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.
[37] Andreas Maurer,et al. A Note on the PAC Bayesian Theorem , 2004, ArXiv.
[38] Shiliang Sun,et al. PAC-Bayes bounds for stable algorithms with instance-dependent priors , 2018, NeurIPS.
[39] Peter L. Bartlett,et al. Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..
[40] Christian Borgs,et al. Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes , 2016, Proceedings of the National Academy of Sciences.
[41] Daniel Kifer,et al. Private Convex Empirical Risk Minimization and High-dimensional Regression , 2012, COLT 2012.
[42] Samy Bengio,et al. Understanding deep learning requires rethinking generalization , 2016, ICLR.
[43] David A. McAllester. A PAC-Bayesian Tutorial with A Dropout Bound , 2013, ArXiv.
[44] John Shawe-Taylor,et al. Tighter PAC-Bayes bounds through distribution-dependent priors , 2013, Theor. Comput. Sci..
[45] Alexandre Lacoste,et al. PAC-Bayesian Theory Meets Bayesian Inference , 2016, NIPS.
[46] Carlo Baldassi,et al. Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses. , 2015, Physical review letters.
[47] Peter Grünwald,et al. Fast Rates for General Unbounded Loss Functions: From ERM to Generalized Bayes , 2016, J. Mach. Learn. Res..
[48] Aaron Roth,et al. The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..
[49] Max Welling,et al. Variational Dropout and the Local Reparameterization Trick , 2015, NIPS 2015.
[50] John Langford,et al. Quantitatively tight sample complexity bounds , 2002 .
[51] Cynthia Dwork,et al. Differential Privacy: A Survey of Results , 2008, TAMC.
[52] Pierre Alquier,et al. Simpler PAC-Bayesian bounds for hostile data , 2016, Machine Learning.
[53] Yee Whye Teh,et al. Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics , 2014, J. Mach. Learn. Res..
[54] André Elisseeff,et al. Stability and Generalization , 2002, J. Mach. Learn. Res..
[55] Tong Zhang. From ɛ-entropy to KL-entropy: Analysis of minimum information complexity density estimation , 2006, math/0702653.
[56] Peter Grünwald,et al. Fast Rates with Unbounded Losses , 2016, ArXiv.
[57] John Shawe-Taylor,et al. A PAC analysis of a Bayesian estimator , 1997, COLT '97.
[58] O. Catoni. PAC-BAYESIAN SUPERVISED CLASSIFICATION: The Thermodynamics of Statistical Learning , 2007, 0712.0248.
[59] Stefano Soatto,et al. Information Dropout: learning optimal representations through noise , 2017, ArXiv.