Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks

Previous math word problem solvers following the encoder-decoder paradigm fail to explicitly incorporate essential math symbolic constraints, leading to unexplainable and unreasonable predictions. Herein, we propose Neural-Symbolic Solver (NS-Solver) to explicitly and seamlessly incorporate different levels of symbolic constraints by auxiliary tasks. Our NS-Solver consists of a problem reader to encode problems, a programmer to generate symbolic equations, and a symbolic executor to obtain answers. Along with target expression supervision, our solver is also optimized via 4 new auxiliary objectives to enforce different symbolic reasoning: a) self-supervised number prediction task predicting both number quantity and number locations; b) commonsense constant prediction task predicting what prior knowledge (e.g. how many legs a chicken has) is required; c) program consistency checker computing the semantic loss between predicted equation and target equation to ensure reasonable equation mapping; d) duality exploiting task exploiting the quasi duality between symbolic equation generation and problem’s part-of-speech generation to enhance the understanding ability of a solver. Besides, to provide a more realistic and challenging benchmark for developing a universal and scalable solver, we also construct a new largescale MWP benchmark CM17K consisting of 4 kinds of MWPs (arithmetic, one-unknown linear, one-unknown non-linear, equation set) with more than 17K samples. Extensive experiments on Math23K and our CM17k demonstrate the superiority of our NS-Solver compared to state-of-the-art methods1. ∗Corresponding Author The code and the new CM17k dataset are available at https://github.com/QinJinghui/NS-Solver.

[1]  Quoc V. Le,et al.  Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension , 2020, ICLR.

[2]  Luke S. Zettlemoyer,et al.  Learning to Automatically Solve Algebra Word Problems , 2014, ACL.

[3]  Dawn Song,et al.  Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty , 2019, NeurIPS.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Richard Socher,et al.  Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning , 2018, ArXiv.

[6]  Wang Ling,et al.  Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems , 2017, ACL.

[7]  Percy Liang,et al.  Data Recombination for Neural Semantic Parsing , 2016, ACL.

[8]  Chen Liang,et al.  Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision , 2016, ACL.

[9]  Jinlan Fu,et al.  A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving , 2020, EMNLP.

[10]  Bolei Zhou,et al.  Visual Question Generation as Dual Task of Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Chitta Baral,et al.  Learning To Use Formulas To Solve Simple Arithmetic Problems , 2016, ACL.

[12]  Tao Qin,et al.  Question Answering and Question Generation as Dual Tasks , 2017, ArXiv.

[13]  Shuming Shi,et al.  Learning Fine-Grained Expressions to Solve Math Word Problems , 2017, EMNLP.

[14]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[15]  Xiaodan Liang,et al.  Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems , 2020, EMNLP.

[16]  Shuming Shi,et al.  Deep Neural Solver for Math Word Problems , 2017, EMNLP.

[17]  Mirella Lapata,et al.  Language to Logical Form with Neural Attention , 2016, ACL.

[18]  Wanxiang Che,et al.  Revisiting Pre-Trained Models for Chinese Natural Language Processing , 2020, FINDINGS.

[19]  Heng Tao Shen,et al.  MathDQN: Solving Arithmetic Word Problems via Deep Reinforcement Learning , 2018, AAAI.

[20]  Yan Wang,et al.  Translating a Math Word Problem to a Expression Tree , 2018, EMNLP.

[21]  Dan Roth,et al.  Mapping to Declarative Knowledge for Word Problem Solving , 2017, TACL.

[22]  Tie-Yan Liu,et al.  Dual Learning for Machine Translation , 2016, NIPS.

[23]  Chen Liang,et al.  Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing , 2018, NeurIPS.

[24]  Ren Yun,et al.  Frame-Based Calculus of Solving Arithmetic Multi-Step Addition and Subtraction Word Problems , 2010, 2010 Second International Workshop on Education Technology and Computer Science.

[25]  Ee-Peng Lim,et al.  Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem , 2020, IJCAI.

[26]  Jing Liu,et al.  Neural Math Word Problem Solver with Reinforcement Learning , 2018, COLING.

[27]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[28]  Eric P. Xing,et al.  GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning , 2021, FINDINGS.

[29]  Zhipeng Xie,et al.  A Goal-Driven Tree-Structured Neural Model for Math Word Problems , 2019, IJCAI.

[30]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[31]  Alexei A. Efros,et al.  Test-Time Training with Self-Supervision for Generalization under Distribution Shifts , 2019, ICML.

[32]  Song-Chun Zhu,et al.  SMART: A Situation Model for Algebra Story Problems via Attributed Grammar , 2020, AAAI.

[33]  Keh-Yih Su,et al.  A Meaning-Based Statistical English Math Word Problem Solver , 2018, NAACL.

[34]  Lipu Zhou,et al.  Learn to Solve Algebra Word Problems Using Quadratic Programming , 2015, EMNLP.

[35]  Yan Wang,et al.  Graph-to-Tree Learning for Solving Math Word Problems , 2020, ACL.

[36]  Wei-Ying Ma,et al.  How well do Computers Solve Math Word Problems? Large-Scale Dataset Construction and Evaluation , 2016, ACL.

[37]  Yefim Bakman,et al.  Robust Understanding of Word Problems with Extraneous Information , 2007, math/0701393.

[38]  Yun-Nung Chen,et al.  Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems , 2018, NAACL.

[39]  Shuming Shi,et al.  Automatically Solving Number Word Problems by Semantic Parsing and Reasoning , 2015, EMNLP.

[40]  Cheqing Jin,et al.  Solving Math Word Problems with Multi-Encoders and Multi-Decoders , 2020, COLING.

[41]  Feng Wang,et al.  Dual Ask-Answer Network for Machine Reading Comprehension , 2018, ArXiv.

[42]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[43]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[44]  Oren Etzioni,et al.  Parsing Algebraic Word Problems into Equations , 2015, TACL.

[45]  Heng Tao Shen,et al.  Template-Based Math Word Problem Solvers with Recursive Neural Networks , 2019, AAAI.

[46]  Song-Chun Zhu,et al.  Learning by Fixing: Solving Math Word Problems with Weak Supervision , 2020, AAAI.

[47]  Dan Roth,et al.  Solving General Arithmetic Word Problems , 2016, EMNLP.

[48]  Nenghai Yu,et al.  Dual Supervised Learning , 2017, ICML.

[49]  Dan Roth,et al.  Unit Dependency Graph and Its Application to Arithmetic Word Problem Solving , 2016, AAAI.

[50]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[51]  Xin Xia,et al.  Code Generation as a Dual Task of Code Summarization , 2019, NeurIPS.