Search Challenges in Natural Language Generation with Complex Optimization Objectives

AbstractAutomatic natural language generation (NLG) is a difficult problem already when merely trying to come up with natural-sounding utterances. Ubiquituous applications, in particular companion technologies, pose the additional challenge of flexible adaptation to a user or a situation. This requires optimizing complex objectives such as information density, in combinatorial search spaces described using declarative input languages. We believe that AI search and planning is a natural match for these problems, and could substantially contribute to solving them effectively. We illustrate this using a concrete example NLG framework, give a summary of the relevant optimization objectives, and provide an initial list of research challenges.

[1]  Michael White,et al.  Minimal Dependency Length in Realization Ranking , 2012, EMNLP.

[2]  Frank Keller,et al.  Incremental, Predictive Parsing with Psycholinguistically Motivated Tree-Adjoining Grammar , 2013, CL.

[3]  S. Edelkamp.  Planning with Pattern Databases , 2014 .

[4]  Walter Kintsch,et al.  Reading rate and retention as a function of the number of propositions in the base structure of sentences , 1973 .

[5]  Michael White,et al.  Learning to Say It Well: Reranking Realizations by Predicted Synthesis Quality , 2006, ACL.

[6]  Kenneth L. McMillan,et al.  Using Unfoldings to Avoid the State Explosion Problem in the Verification of Asynchronous Circuits , 1992, CAV.

[7]  Daniel Gildea,et al.  Do Grammars Minimize Dependency Length? , 2010, Cogn. Sci..

[8]  Antti Valmari,et al.  Stubborn sets for reduced state space generation , 1991, Applications and Theory of Petri Nets.

[9]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[10]  Frank A. Drews,et al.  Passenger and Cell-Phone Conversations in Simulated Driving , 2004 .

[11]  Patrik Haslum,et al.  Admissible Heuristics for Optimal Planning , 2000, AIPS.

[12]  Michael White,et al.  Reining in CCG Chart Realization , 2004, INLG.

[13]  J. Kleer,et al.  Heuristic Search for Target-Value Path Problem , 2008 .

[14]  Josef van Genabith,et al.  Robust PCFG-Based Generation Using Automatically Acquired LFG Approximations , 2006, ACL.

[15]  Malte Helmert,et al.  Efficient Stubborn Sets: Generalized Algorithms and Selection Strategies , 2014, ICAPS.

[16]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[17]  E. Gibson.  Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[18]  Michael White,et al.  Perceptron Reranking for CCG Realization , 2009, EMNLP.

[19]  Roger Levy,et al.  Speakers optimize information density through syntactic reduction , 2006, NIPS.

[20]  Martin Kay,et al.  Chart Generation , 1996, ACL.

[21]  Vera Demberg,et al.  Linguistic cognitive load : implications for automotive UIs , 2011 .

[22]  T. Florian Jaeger,et al.  Redundancy and reduction: Speakers manage syntactic information density , 2010, Cognitive Psychology.

[23]  Michael White,et al.  Efficient Realization of Coordinate Structures in Combinatory Categorial Grammar , 2006 .

[24]  Roni Stern,et al.  Solving the Target-Value Search Problem , 2014, SOCS.

[25]  Patrik Haslum,et al.  Merge-and-Shrink Abstraction , 2014, J. ACM.

[26]  Michael White,et al.  Linguistically Motivated Complementizer Choice in Surface Realization , 2011 .

[27]  Patrik Haslum,et al.  Improving Delete Relaxation Heuristics Through Explicitly Represented Conjunctions , 2014, J. Artif. Intell. Res..

[28]  Matthew W. Crocker,et al.  Information Density and Linguistic Encoding (IDeaL) , 2015, KI - Künstliche Intelligenz.

[29]  S. Frank,et al.  The ERP response to the amount of information conveyed by words in sentences , 2015, Brain and Language.

[30]  Stephan Oepen,et al.  High Efficiency Realization for a Wide-Coverage Unification Grammar , 2005, IJCNLP.

[31]  Michael White,et al.  Better Surface Realization through Psycholinguistics , 2014, Lang. Linguistics Compass.

[32]  Helen F. Hastie,et al.  Optimising Incremental Dialogue Decisions Using Information Density for Interactive Systems , 2012, EMNLP-CoNLL.

[33]  David Crundall,et al.  Regulating Conversation During Driving: A Problem for Mobile Telephones? , 2005 .

[34]  R. Levy.  Expectation-based syntactic comprehension , 2008, Cognition.

[35]  Charles F. Hockett,et al.  A mathematical theory of communication , 1948, MOCO.

[36]  Albert Gatt,et al.  Proceedings of the UCNLG+Eval: Language Generation and Evaluation Workshop , 2011 .

[37]  Michael White,et al.  Designing Agreement Features for Realization Ranking , 2010, COLING.

[38]  Jörg Hoffmann,et al.  "Distance"? Who Cares? Tailoring Merge-and-Shrink Heuristics to Detect Unsolvability , 2014, ECAI.

[39]  Patrik Haslum,et al.  Directed Unfolding of Petri Nets , 2008, Trans. Petri Nets Other Model. Concurr..

[40]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.