On Learning Sparse Boolean Formulae for Explaining AI Decisions

In this paper, we consider the problem of learning Boolean formulae from examples obtained by actively querying an oracle that can label these examplesz as either positive or negative. This problem has received attention in both machine learning as well as formal methods communities, and it has been shown to have exponential worst-case complexity in the general case as well as for many restrictions. In this paper, we focus on learning sparse Boolean formulae which depend on only a small (but unknown) subset of the overall vocabulary of atomic propositions. We propose an efficient algorithm to learn these sparse Boolean formulae with a given confidence. This assumption of sparsity is motivated by the problem of mining explanations for decisions made by artificially intelligent (AI) algorithms, where the explanation of individual decisions may depend on a small but unknown subset of all the inputs to the algorithm. We demonstrate the use of our algorithm in automatically generating explanations of these decisions. These explanations will make intelligent systems more understandable and accountable to human users, facilitate easier audits and provide diagnostic information in the case of failure. The proposed approach treats the AI algorithm as a black-box oracle; hence, it is broadly applicable and agnostic to the specific AI algorithm. We illustrate the practical effectiveness of our approach on a diverse set of case studies.

[1]  Dana Angluin,et al.  Computational learning theory: survey and selected bibliography , 1992, STOC '92.

[2]  Ankur Taly,et al.  Automated synthesis of symbolic instruction encodings from I/O samples , 2012, PLDI.

[3]  Hadas Kress-Gazit,et al.  Sorry Dave, I'm Afraid I Can't Do That: Explaining Unachievable Robot Tasks Using Natural Language , 2013, Robotics: Science and Systems.

[4]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[5]  N Moray,et al.  Trust, control strategies and allocation of function in human-machine systems. , 1992, Ergonomics.

[6]  Henny B. Sipma,et al.  Constructing invariants for hybrid systems , 2004, Formal Methods Syst. Des..

[7]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[8]  Daniel Kroening,et al.  Ranking function synthesis for bit-vector relations , 2010, Formal Methods Syst. Des..

[9]  Leslie G. Valiant,et al.  Learning Boolean formulas , 1994, JACM.

[10]  Patrice Godefroid,et al.  Automatic Synthesis of Specifications from the Dynamic Observation of Reactive Programs , 1997, TACAS.

[11]  Sanjit A. Seshia,et al.  A theory of formal synthesis via inductive learning , 2015, Acta Informatica.

[12]  Ashish Tiwari,et al.  Synthesis of optimal switching logic for hybrid systems , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[13]  Balázs Szörényi,et al.  Theory revision with queries: Horn, read-once, and parity formulas , 2004, Artif. Intell..

[14]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[15]  Erik Strumbelj,et al.  Explaining prediction models and individual predictions with feature contributions , 2014, Knowledge and Information Systems.

[16]  Sumit Gulwani,et al.  Oracle-guided component-based program synthesis , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[17]  Sriram Sankaranarayanan,et al.  Automatic invariant generation for hybrid systems using ideal fixed points , 2010, HSCC '10.

[18]  Luis Enrique Sucar,et al.  Generating Explanations Based on Markov Decision Processes , 2009, MICAI.

[19]  Rocco A. Servedio,et al.  On PAC learning algorithms for rich Boolean function classes , 2006, Theoretical Computer Science.

[20]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[21]  John-Jules Ch. Meyer,et al.  Explaining Simulations Through Self Explaining Agents , 2010, J. Artif. Soc. Soc. Simul..

[22]  Joao Marques-Silva,et al.  Synthesizing Safe Bit-Precise Invariants , 2014, TACAS.

[23]  Arie Gurfinkel,et al.  Synthesizing Ranking Functions from Bits and Pieces , 2016, TACAS.

[24]  Yishay Mansour,et al.  Learning Boolean Functions via the Fourier Transform , 1994 .

[25]  Domagoj Babic,et al.  Sigma*: symbolic learning of input-output specifications , 2013, POPL.

[26]  Paolo Traverso,et al.  Automated Planning: Theory & Practice , 2004 .

[27]  Changhe Yuan,et al.  Most Relevant Explanation in Bayesian Networks , 2011, J. Artif. Intell. Res..

[28]  Abraham Silberschatz,et al.  Learning and verifying quantified boolean queries by example , 2013, PODS '13.

[29]  Paolo Traverso,et al.  Automated planning - theory and practice , 2004 .

[30]  Viktor Kuncak,et al.  Counterexample-Guided Quantifier Instantiation for Synthesis in SMT , 2015, CAV.

[31]  Marco Bozzano,et al.  Towards Pareto-optimal parameter synthesis for monotonie cost functions , 2014, 2014 Formal Methods in Computer-Aided Design (FMCAD).

[32]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[33]  Christopher Miller,et al.  A model-based approach to synthesizing insulin infusion pump usage parameters for diabetic patients , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[34]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[35]  Vasumathi Raman.  Reactive switching protocols for multi-robot high-level tasks , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.