Joint learning of constraint weights and gradient inputs in Gradient Symbolic Computation with constrained optimization

This paper proposes a method for the joint optimization of constraint weights and symbol activations within the Gradient Symbolic Computation (GSC) framework. The set of grammars representable in GSC is proven to be a subset of those representable with lexically-scaled faithfulness constraints. This fact is then used to recast the problem of learning constraint weights and symbol activations in GSC as a quadratically-constrained version of learning lexically-scaled faithfulness grammars. This results in an optimization problem that can be solved using Sequential Quadratic Programming.

[1]  Y. Morin.  La liaison relève-t-elle d'une tendance à éviter les hiatus ? Réflexions sur son évolution historique , 2005 .

[2]  Matthew Goldrick,et al.  Optimization and Quantization in Gradient Symbol Systems: A Framework for Integrating the Continuous and the Discrete in Cognition , 2014, Cogn. Sci..

[3]  Joe Pater,et al.  Gradient Exceptionality in Maximum Entropy Grammar with Lexically Specific Constraints , 2016 .

[4]  Paul Boersma,et al.  Gradual constraint-ranking learning algorithm predicts acquisition order , 1999 .

[5]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[6]  Anne-Michelle Tessier,et al.  Gradual Learning and Faithfulness: Consequences of Ranked vs. Weighted Constraints * , 2008 .

[7]  Gaja Jarosz,et al.  Learning Exceptionality and Variation with Lexically Scaled MaxEnt , 2019 .

[8]  Bruce Hayes,et al.  A Maximum Entropy Model of Phonotactics and Phonotactic Learning , 2008, Linguistic Inquiry.

[9]  Gerhard Jäger,et al.  Maximum Entropy Models and Stochastic Optimality Theory , 2003 .

[10]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[11]  P. Smolensky,et al.  Gradient Symbolic Representations in Grammar: The case of French Liaison , 2016 .

[12]  Eric Bakovic,et al.  Phonological opacity as local optimization in Gradient Symbolic Computation , 2018 .

[13]  Pyeong Whan Cho,et al.  Incremental parsing in a continuous dynamical system: sentence processing in Gradient Symbolic Computation , 2017 .

[14]  P. Boersma,et al.  Convergence Properties of a Gradual Learning Algorithm for Harmonic Grammar , 2013 .

[15]  Diana Apoussidou,et al.  The learnability of metrical phonology , 2007 .

[16]  Tal Linzen,et al.  Lexical and phonological variation in Russian prepositions* , 2013, Phonology.

[17]  Brian W. Smith.  Phonologically Conditioned Allomorphy and UR Constraints , 2015 .

[18]  Eric R Rosen,et al.  Learning complex inflectional paradigms through blended gradient inputs , 2019 .

[19]  Bernard Tranel,et al.  French liaison and elision revisited: A unified account within Optimality Theory , 1994 .

[20]  Karen Jesney,et al.  Biases in Harmonic Grammar: the road to restrictive learning , 2011 .

[21]  Eric Rosen,et al.  Predicting the unpredictable : Capturing the apparent semi-regularity of rendaku voicing in Japanese through harmonic grammar ∗ , 2016 .

[22]  Eric Rosen,et al.  Learning a gradient grammar of French liaison , 2020 .

[23]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[24]  Mark Johnson,et al.  Learning OT constraint rankings using a maximum entropy model , 2003 .