Probabilistic Generation of Weather Forecast Texts

This paper reports experiments in which pC RU — a generation framework that combines probabilistic generation methodology with a comprehensive model of the generation space — is used to semi-automatically create several versions of a weather forecast text generator. The generators are evaluated in terms of output quality, development time and computational efficiency against (i) human forecasters, (ii) a traditional handcrafted pipelined NLG system, and (iii) a HALOGEN-style statistical generator. The most striking result is that despite acquiring all decision-making abilities automatically, the best pC RU generators receive higher scores from human judges than forecasts written by experts.

[1]  George R. Doddington,et al.  Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics , 2002 .

[2]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[3]  Anja Belz,et al.  Comparing Automatic and Human Evaluation of NLG Systems , 2006, EACL.

[4]  Jim Hunter,et al.  Choosing words in computer-generated weather forecasts , 2005, Artif. Intell..

[5]  Irene Langkilde-Geary.  An Exploratory Application of Constraint Optimization in Mozart to Probabilistic Natural Language Processing , 2004, CSLP.

[6]  Ehud Reiter,et al.  Has a Consensus NL Generation Architecture Appeared, and is it Psycholinguistically Plausible? , 1994, INLG.

[7]  Anja Belz,et al.  Context-Free Representational Underspecification for NLG , 2004 .

[8]  Andreas Stolcke,et al.  SRILM - an extensible language modeling toolkit , 2002, INTERSPEECH.

[9]  S. Oepen,et al.  Paraphrasing Treebanks for Stochastic Realization Ranking , 2004 .

[10]  Kevin Knight,et al.  Generation that Exploits Corpus-Based Statistical Knowledge , 1998, ACL.

[11]  Roger Evans,et al.  Empirically-based Control of Natural Language Generation , 2005, ACL.

[12]  Richard Power.  Planning texts by constraint satisfaction , 2000, COLING.

[13]  Michael White,et al.  Reining in CCG Chart Realization , 2004, INLG.

[14]  Anja Belz,et al.  Statistical Generation: Three Methods Compared and Evaluated , 2005, ENLG.

[15]  M. Strube,et al.  Using an Annotated Corpus As a Knowledge Source For Language Generation , 2005 .

[16]  Eduard Hovy,et al.  Generating Natural Language Under Pragmatic Constraints , 1988 .

[17]  Liang Huang,et al.  Statistical Syntax-Directed Translation with Extended Domain of Locality , 2006, AMTA.

[18]  Josef van Genabith,et al.  Robust PCFG-Based Generation Using Automatically Acquired LFG Approximations , 2006, ACL.