论文信息 - Teaching Uncertainty Quantification in Machine Learning through Use Cases

Teaching Uncertainty Quantification in Machine Learning through Use Cases

Uncertainty in machine learning is not generally taught as general knowledge in Machine Learning course curricula. In this paper we propose a short curriculum for a course about uncertainty in machine learning, and complement the course with a selection of use cases, aimed to trigger discussion and let students play with the concepts of uncertainty in a programming setting. Our use cases cover the concept of output uncertainty, Bayesian neural networks and weight distributions, sources of uncertainty, and out of distribution detection. We expect that this curriculum and set of use cases motivates the community to adopt these important concepts into courses for safety in AI.

Matias Valdenegro-Toro | Matias Valdenegro-Toro

[1] Christopher K. I. Williams,et al. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[2] Stephen E. Fienberg,et al. The Comparison and Evaluation of Forecasters. , 1983 .

[3] Charles Blundell,et al. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[4] Naveen Garg,et al. DropConnect is effective in modeling uncertainty of Bayesian deep networks , 2019, Scientific Reports.

[5] L. E. Lynn,et al. Teaching and Learning with Cases: A Guidebook , 1999 .

[6] Xiao Xiang Zhu,et al. A survey of uncertainty in deep neural networks , 2021, Artificial Intelligence Review.

[7] Michael Betancourt,et al. A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[8] Zoubin Ghahramani,et al. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[9] A. Kiureghian,et al. Aleatory or epistemic? Does it matter? , 2009 .

[10] Alex Kendall,et al. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[11] Kilian Q. Weinberger,et al. On Calibration of Modern Neural Networks , 2017, ICML.

[12] Matias Valdenegro-Toro,et al. I Find Your Lack of Uncertainty in Computer Vision Disturbing , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).