Deep Reinforcement Learning Meets Structured Prediction

s (11): Abstract 2: A New Perspective on Adversarial Perturbations in Debugging Machine Learning Models, Madry 08:00 AM2: A New Perspective on Adversarial Perturbations in Debugging Machine Learning Models, Madry 08:00 AM The widespread susceptibility of the current ML models to adversarial perturbations is an intensely studied but still mystifying phenomenon. A popular view is that these perturbations are aberrations that arise due to statistical fluctuations in the training data and/or high-dimensional nature of our inputs. But is this really the case? In this talk, I will present a new perspective on the phenomenon of adversarial perturbations. This perspective ties this phenomenon to the existence of "non-robust" features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans. Such patterns turn out to be prevalent in our real-world datasets and also shed light on previously observed phenomena in adversarial robustness, including transferability of adversarial examples and properties of robust models. Finally, this perspective suggests that we may need to recalibrate our expectations in terms of how models should make their decisions, and how we should interpret them. Abstract 3: Similarity of Neural Network Representations Revisited in Debugging Machine Learning Models, Kornblith 08:30 AM Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We introduce a similarity index that measures the relationship between representational similarity matrices. We show that this similarity index is equivalent to centered kernel alignment (CKA) and analyze its relationship to canonical correlation analysis. Unlike other methods, CKA can reliably identify correspondences between representations of layers in networks trained from different initializations. Moreover, CKA can reveal network pathology that is not evident from test accuracy alone.3: Similarity of Neural Network Representations Revisited in Debugging Machine Learning Models, Kornblith 08:30 AM Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We introduce a similarity index that measures the relationship between representational similarity matrices. We show that this similarity index is equivalent to centered kernel alignment (CKA) and analyze its relationship to canonical correlation analysis. Unlike other methods, CKA can reliably identify correspondences between representations of layers in networks trained from different initializations. Moreover, CKA can reveal network pathology that is not evident from test accuracy alone. Abstract 6: Verifiable Reinforcement Learning via Policy Extraction in Debugging Machine Learning Models, Bastani 09:10 AM6: Verifiable Reinforcement Learning via Policy Extraction in Debugging Machine Learning Models, Bastani 09:10 AM While deep reinforcement learning has successfully solved many challenging control tasks, its real-world applicability has been limited by the inability to ensure the safety of learned policies. We propose VIPER, an approach to verifiable reinforcement learning by training decision tree policies, which can represent complex policies (since they are nonparametric), yet can be efficiently verified using existing techniques (since they are highly structured). We use VIPER to learn a decision tree policy for a toy game based on Pong that provably never loses. Abstract 7: Debugging Machine Learning via Model Assertions in Debugging Machine Learning Models, Kang 09:40 AM7: Debugging Machine Learning via Model Assertions in Debugging Machine Learning Models, Kang 09:40 AM Machine learning models are being deployed in mission-critical settings, such as self-driving cars. However, these models can fail in complex ways, so it is imperative that application developers find ways to debug these models. We propose adapting software assertions, or boolean statements about the state of a program that must be true, to the task of debugging ML models. With model assertions, ML developers can specify constraints on model outputs, e.g., cars should not disappear and reappear in successive frames of a video. We propose several ways to use model assertions in ML debugging, including use in runtime monitoring, in performing corrective actions, and in collecting “hard examples” to further train models with human labeling or weak supervision. We show that, for a video analytics task, simple assertions can effectively find errors and correction rules can effectively correct model output (up to 100% and 90% respectively). We additionally collect and label parts of video where assertions fire (as a form of active ICLR 2019 Workshop book Generated Thu Sep 10, 2020 Page 8 of 20 learning) and show that this procedure can improve model performance by up to 2×. Abstract 10: Discovering Natural Bugs Using Adversarial Data Perturbations in Debugging Machine Learning Models, Singh 10:10 AM10: Discovering Natural Bugs Using Adversarial Data Perturbations in Debugging Machine Learning Models, Singh 10:10 AM Determining when a machine learning model is “good enough” is challenging since held-out accuracy metrics significantly overestimate real-world performance. In this talk, I will describe automated techniques to detect bugs that can occur naturally when a model is deployed. I will start by approaches to identify “semantically equivalent” adversaries that should not change the meaning of the input, but lead to a change in the model’s predictions. Then I will present our work on evaluating the consistency behavior of the model by exploring performance on new instances that are “implied” by the model’s predictions. I will also describe a method to understand and debug models by adversarially modifying the training data to change the model’s predictions. The talk will include applications of these ideas on a number of NLP tasks, such as reading comprehension, visual QA, and knowledge graph completion. Abstract 11: "Debugging" Discriminatory ML Systems in Debugging Machine Learning Models, Raji 10:40 AM11: "Debugging" Discriminatory ML Systems in Debugging Machine Learning Models, Raji 10:40 AM If a machine learning (ML) model is illegally discriminatory towards vulnerable and underrepresented populations, can we really say it works? Of course not! That illegal behaviour negates the functionality of the ML model, just as much as overfitting or other typically acknowledged ML "bugs". This talk explores the redefinition of what it means for a model to "work" well enough to deploy and dives into the analogy of software engineering debugging practice to explain current strategies for diagnosing, reporting, addressing and preventing the further development of discriminatory ML models. Abstract 12: NeuralVerification.jl: Algorithms for Verifying Deep Neural Networks in Debugging Machine Learning Models, Arnon, Lazarus 11:00 AM12: NeuralVerification.jl: Algorithms for Verifying Deep Neural Networks in Debugging Machine Learning Models, Arnon, Lazarus 11:00 AM Deep neural networks (DNNs) are widely used for nonlinear function approximation with applications ranging from computer vision to control. Although DNNs involve the composition of simple arithmetic operations, it can be very challenging to verify whether a particular network satisfies certain input-output properties. This work introduces NeuralVerification.jl, a software package that implements methods that have emerged recently for soundly verifying such properties. These methods borrow insights from reachability analysis, optimization, and search. We present the formal problem definition and briefly discuss the fundamental differences between the implemented algorithms. In addition, we provide a pedagogical example of how to use the library. Abstract 15: Safe and Reliable Machine Learning: Preventing and Identifying Failures in Debugging Machine Learning Models, Saria 01:30 PM15: Safe and Reliable Machine Learning: Preventing and Identifying Failures in Debugging Machine Learning Models, Saria 01:30 PM Machine Learning driven decision-making systems are being increasingly used to decide bank loans, make hiring decisions,perform clinical decision-making, and more. As we march towards a future in which these systems underpin most of society’s decision-making infrastructure, it is critical for us to understand the principles that will help us engineer for reliability. Drawing from reliability engineering, we will briefly outline three principles to group and guide technical solutions for addressing and ensuring reliability in machine learning systems: 1) Failure Prevention, 2) Failure Identification, and 3) Maintenance. In particular, we will discuss a framework (https://arxiv.org/abs/1904.07204) for preventing failures due to differences between the training and deployment environments that proactively addresses the problem of dataset shift. We will contrast this with typical reactive solutions which require deployment environment data and discuss relations with similar problems such as robustness to adversarial examples. Abstract 16: Better Code for Less Debugging with AutoGraph in Debugging Machine Learning Models, Moldovan 02:00 PM16: Better Code for Less Debugging with AutoGraph in Debugging Machine Learning Models, Moldovan 02:00 PM The fast-paced nature of machine learning research and development, with many ideas advancing rapidly from research to production, puts it at increased risk of programming errors, which can be particularly insidious when combined with machine learning. In this talk we discuss defensive design as a way to reduce the chance for such errors to occur in the first place, and present AutoGraph, a tool which facilitates defensive design by allowing more legible code that is still efficient and portabl