Neuro-Symbolic Procedural Planning with Commonsense Prompting

Procedural planning aims to implement complex high-level goals by decomposition into sequential simpler low-level steps. Although procedural planning is a basic skill set for humans in daily life, it remains a challenge for large language models (LLMs) that lack a deep understanding of the cause-effect relations in procedures. Previous methods require manual exemplars to acquire procedural planning knowledge from LLMs in the zero-shot setting. However, such elicited pre-trained knowledge in LLMs induces spurious correlations between goals and steps, which impair the model generalization to unseen tasks. In contrast, this paper proposes a neuro-symbolic procedural PLANner (PLAN) that elicits procedural planning knowledge from the LLMs with commonsense-infused prompting. To mitigate spurious goal-step correlations, we use symbolic program executors on the latent procedural representations to formalize prompts from commonsense knowledge bases as a causal intervention toward the Structural Causal Model. Both automatic and human evaluations on WikiHow and RobotHow show the superiority of PLAN on procedural planning without further training or manual exemplars.

[1]  Michael R Douglas Large Language Models , 2023, Commun. ACM.

[2]  S. Levine,et al.  Do As I Can, Not As I Say: Grounding Language in Robotic Affordances , 2022, CoRL.

[3]  Le Sun,et al.  Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View , 2022, ACL.

[4]  Jey Han Lau,et al.  An Interpretable Neuro-Symbolic Reasoning Framework for Task-Oriented Dialogue Generation , 2022, ACL.

[5]  Chen Change Loy,et al.  Conditional Prompt Learning for Vision-Language Models , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Dale Schuurmans,et al.  Chain of Thought Prompting Elicits Reasoning in Large Language Models , 2022, NeurIPS.

[7]  Huajun Chen,et al.  Ontology-enhanced Prompt-tuning for Few-shot Learning , 2022, WWW.

[8]  Zhiting Hu,et al.  A Causal Lens for Controllable Text Generation , 2022, NeurIPS.

[9]  P. Abbeel,et al.  Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents , 2022, ICML.

[10]  Jie Zhou,et al.  On Transferability of Prompt Tuning for Natural Language Processing , 2021, NAACL.

[11]  R. Weischedel,et al.  Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals , 2021, ACL.

[12]  Shafiq R. Joty,et al.  GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems , 2021, ACL.

[13]  Devendra Singh Chaplot,et al.  FILM: Following Instructions in Language with Modular Methods , 2021, ICLR.

[14]  Michael Saxon,et al.  Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer , 2021, AAAI.

[15]  Le Song,et al.  ProTo: Program-Guided Transformer for Program-Guided Tasks , 2021, NeurIPS.

[16]  Dilek Z. Hakkani-Tür,et al.  TEACh: Task-driven Embodied Agents that Chat , 2021, AAAI.

[17]  Chris Callison-Burch,et al.  Goal-Oriented Script Construction , 2021, INLG.

[18]  Dieter Fox,et al.  A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution , 2021, CoRL.

[19]  Joshua B. Tenenbaum,et al.  Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning , 2021, NeurIPS.

[20]  Chuanqi Tan,et al.  KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction , 2021, WWW.

[21]  Alexander M. Rush,et al.  How many data points is a prompt worth? , 2021, NAACL.

[22]  Yaohui Jin,et al.  Multi-Task Learning for Logically Dependent Tasks from the Perspective of Causal Inference , 2020, EMNLP.

[23]  Peter Alexander Jansen Visually-Grounded Planning without Vision: Language Models Infer Detailed Plans from High-level Instructions , 2020, FINDINGS.

[24]  Xian-Sheng Hua,et al.  Interventional Few-Shot Learning , 2020, NeurIPS.

[25]  Chris Callison-Burch,et al.  Reasoning about Goals, Steps, and Temporal Ordering with WikiHow , 2020, EMNLP.

[26]  Chris Callison-Burch,et al.  Intent Detection with WikiHow , 2020, AACL.

[27]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[28]  Uri Shalit,et al.  CausaLM: Causal Model Explanation Through Counterfactual Language Models , 2020, CL.

[29]  Katherine A. Keith,et al.  Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates , 2020, ACL.

[30]  Joseph J. Lim,et al.  Program Guided Agent , 2020, ICLR.

[31]  Rachel Rudinger,et al.  Causal Inference of Script Knowledge , 2020, EMNLP.

[32]  Luke Zettlemoyer,et al.  ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  B. Schölkopf,et al.  Causality for Machine Learning , 2019, Probabilistic and Causal Inference.

[34]  Omer Levy,et al.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension , 2019, ACL.

[35]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[36]  Juan Carlos Niebles,et al.  Procedure Planning in Instructional Videos , 2019, ECCV.

[37]  J. Tenenbaum,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision , 2019, ICLR.

[38]  Kilian Q. Weinberger,et al.  BERTScore: Evaluating Text Generation with BERT , 2019, ICLR.

[39]  Hong Li,et al.  Identification and transformation difficulty in problem solving: Electrophysiological evidence from chunk decomposition , 2019, Biological Psychology.

[40]  William Yang Wang,et al.  WikiHow: A Large Scale Text Summarization Dataset , 2018, ArXiv.

[41]  Chuang Gan,et al.  Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding , 2018, NeurIPS.

[42]  Mark Dredze,et al.  Challenges of Using Text Classifiers for Causal Inference , 2018, EMNLP.

[43]  Stefan Ultes,et al.  MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling , 2018, EMNLP.

[44]  Sanja Fidler,et al.  VirtualHome: Simulating Household Activities Via Programs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Claudiu Musat,et al.  Goal-Oriented Chatbot Dialog Management Bootstrapping with Transfer Learning , 2018, IJCAI.

[46]  Bernhard Schölkopf,et al.  Elements of Causal Inference: Foundations and Learning Algorithms , 2017 .

[47]  Christopher D. Manning,et al.  Key-Value Retrieval Networks for Task-Oriented Dialogue , 2017, SIGDIAL Conference.

[48]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[49]  C. Zheng,et al.  Causal mediation analysis in the context of clinical research. , 2016, Annals of translational medicine.

[50]  Lilian D. A. Wanzare,et al.  A Crowdsourced Database of Event Sequence Descriptions for the Acquisition of High-quality Script Knowledge , 2016, LREC.

[51]  Matt J. Kusner,et al.  From Word Embeddings To Document Distances , 2015, ICML.

[52]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[53]  Manfred Pinkal,et al.  Learning Script Knowledge with Web Experiments , 2010, ACL.

[54]  Rakesh Gupta,et al.  Common Sense Data Acquisition for Indoor Mobile Robots , 2004, AAAI.

[55]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[56]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[57]  J. Pine,et al.  Chunking mechanisms in human learning , 2001, Trends in Cognitive Sciences.

[58]  S. Ohlsson,et al.  Constraint relaxation and chunk decomposition in insight problem solving , 1999 .

[59]  Douglas J. Pearson Learning Procedural Planning Knowledge in Complex Environments , 1996, AAAI/IAAI, Vol. 2.

[60]  J. Pearl Causal diagrams for empirical research , 1995 .

[61]  Illtyd Trethowan Causality , 1938 .

[62]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[63]  J. Böhnke Explanation in causal inference: Methods for mediation and interaction. , 2016, Quarterly journal of experimental psychology.

[64]  Petra Theunissen,et al.  Conference Paper , 2009 .

[65]  Steven M. Smith,et al.  Getting into and out of mental ruts: A theory of fixation, incubation, and insight. , 1995 .

[66]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[67]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .