Explaining Patterns in Data with Language Models via Interpretable Autoprompting

autoprompting (iPrompt), an a natural-language explaining the data. iPrompt iteratively alternates between generating explanations with an LLM and reranking them based on their performance when used as a prompt. Experiments on a wide range of datasets, from synthetic mathematics to natural-language understanding, show that iPrompt can yield meaningful insights by accurately finding groundtruth dataset descriptions. the prompts produced for generalization: on real-world sentiment classification datasets, iPrompt produces prompts that improve upon prompts for Finally, experiments with an dataset show the potential for iPrompt to in discovery. 1

[1]  Alexander M. Rush,et al.  Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models , 2022, IEEE Transactions on Visualization and Computer Graphics.

[2]  Jianfeng Gao,et al.  Emb-GAM: an Interpretable and Efficient Predictor using Pre-trained Language Models , 2022, ArXiv.

[3]  Yuhuai Wu,et al.  Solving Quantitative Reasoning Problems with Language Models , 2022, NeurIPS.

[4]  Xi Victoria Lin,et al.  OPT: Open Pre-trained Transformer Language Models , 2022, ArXiv.

[5]  Kuntal Kumar Pal,et al.  Benchmarking Generalization via In-Context Instructions on 1, 600+ Language Tasks , 2022, ArXiv.

[6]  Stella Rose Biderman,et al.  GPT-NeoX-20B: An Open-Source Autoregressive Language Model , 2022, BIGSCIENCE.

[7]  Cornelia Caragea,et al.  SciNLI: A Corpus for Natural Language Inference on Scientific Text , 2022, ACL.

[8]  Alexander M. Rush,et al.  PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts , 2022, ACL.

[9]  Yan Shuo Tan,et al.  Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods , 2022, ICML.

[10]  D. Klein,et al.  Describing Differences between Text Distributions with Natural Language , 2022, ICML.

[11]  Alexander M. Rush,et al.  Multitask Prompted Training Enables Zero-Shot Task Generalization , 2021, ICLR.

[12]  Ellie Pavlick,et al.  Do Prompt-Based Models Really Understand the Meaning of Their Prompts? , 2021, NAACL.

[13]  Maosong Sun,et al.  Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification , 2021, ACL.

[14]  Hiroaki Hayashi,et al.  Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing , 2021, ACM Comput. Surv..

[15]  Fabio Petroni,et al.  Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models , 2021, FINDINGS.

[16]  S. Riedel,et al.  Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity , 2021, ACL.

[17]  David Bau,et al.  Locating and Editing Factual Knowledge in GPT , 2022, ArXiv.

[18]  Chandan Singh,et al.  Adaptive wavelet distillation from neural networks through interpretations , 2021, NeurIPS.

[19]  Weida Tong,et al.  InferBERT: A Transformer-Based Causal Inference Framework for Enhancing Pharmacovigilance , 2021, Frontiers in Artificial Intelligence.

[20]  Zhiyuan Liu,et al.  PTR: Prompt Tuning with Rules for Text Classification , 2021, AI Open.

[21]  Chandan Singh,et al.  Imodels: a Python Package for Fitting Interpretable Models , 2021, J. Open Source Softw..

[22]  Stella Biderman,et al.  GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow , 2021 .

[23]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[24]  Thomas Lukasiewicz,et al.  Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration , 2020, AAAI.

[25]  Percy Liang,et al.  Prefix-Tuning: Optimizing Continuous Prompts for Generation , 2021, ACL.

[26]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[27]  Colin Raffel,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[28]  Yejin Choi,et al.  The Curious Case of Neural Text Degeneration , 2019, ICLR.

[29]  Sebastian Riedel,et al.  Language Models as Knowledge Bases? , 2019, EMNLP.

[30]  Richard Socher,et al.  Neural Text Summarization: A Critical Evaluation , 2019, EMNLP.

[31]  Sameer Singh,et al.  Universal Adversarial Triggers for Attacking and Analyzing NLP , 2019, EMNLP.

[32]  Frederick Liu,et al.  Incorporating Priors with Feature Attribution on Text Classification , 2019, ACL.

[33]  Gabriel Erion,et al.  Explainable AI for Trees: From Local Explanations to Global Understanding , 2019, ArXiv.

[34]  Chandan Singh,et al.  Hierarchical interpretations for neural network predictions , 2018, ICLR.

[35]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[36]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[37]  Thomas Lukasiewicz,et al.  e-SNLI: Natural Language Inference with Natural Language Explanations , 2018, NeurIPS.

[38]  Guillaume Lample,et al.  What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties , 2018, ACL.

[39]  Arvind Satyanarayan,et al.  The Building Blocks of Interpretability , 2018 .

[40]  Zhifang Sui,et al.  Table-to-text Generation by Structure-aware Seq2seq Learning , 2017, AAAI.

[41]  Rich Caruana,et al.  Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation , 2017, AIES.

[42]  Yan Liu,et al.  Detecting Statistical Interactions from Neural Network Weights , 2017, ICLR.

[43]  Trevor Darrell,et al.  Generating Visual Explanations , 2016, ECCV.

[44]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[45]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[46]  Jack L. Gallant,et al.  Pycortex: an interactive surface visualizer for fMRI , 2015, Front. Neuroinform..

[47]  Pekka Korhonen,et al.  Good debt or bad debt: Detecting semantic orientations in economic texts , 2013, J. Assoc. Inf. Sci. Technol..

[48]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[49]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[50]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[51]  Jason Eisner,et al.  Modeling Annotators: A Generative Approach to Learning from Annotator Rationales , 2008, EMNLP.

[52]  Heikki Mannila,et al.  Principles of Data Mining , 2001, Undergraduate Topics in Computer Science.

[53]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[54]  Helio J. C. Barbosa,et al.  Symbolic regression via genetic programming , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[55]  C. Cordell Green,et al.  What Is Program Synthesis? , 1985, J. Autom. Reason..

[56]  Zohar Manna,et al.  A Deductive Approach to Program Synthesis , 1979, TOPL.