Learning Scripts as Hidden Markov Models

Scripts have been proposed to model the stereotypical event sequences found in narratives. They can be applied to make a variety of inferences including filling gaps in the narratives and resolving ambiguous references. This paper proposes the first formal framework for scripts based on Hidden Markov Models (HMMs). Our framework supports robust inference and learning algorithms, which are lacking in previous clustering models. We develop an algorithm for structure and parameter learning based on Expectation Maximization and evaluate it on a number of natural datasets. The results show that our algorithm is superior to several informed baselines for predicting missing events in partial observation sequences.

[1]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[2]  Jackie Chi Kit Cheung,et al.  Probabilistic Frame Induction , 2013, NAACL.

[3]  Nathanael Chambers,et al.  Event Schema Induction with a Probabilistic Entity-Driven Model , 2013, EMNLP.

[4]  D. Haussler,et al.  Hidden Markov models in computational biology. Applications to protein modeling. , 1993, Journal of molecular biology.

[5]  Enrique Vidal,et al.  What Is the Search Space of the Regular Inference? , 1994, ICGI.

[6]  Lalit R. Bahl,et al.  A Maximum Likelihood Approach to Continuous Speech Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Gerald DeJong,et al.  Explanation-Based Learning: An Alternative View , 2005, Machine Learning.

[8]  A. Krogh.  Hidden Markov Models in Computational Biology Applications to Protein Modeling UCSC CRL , 1993 .

[9]  Rakesh Gupta,et al.  Common Sense Data Acquisition for Indoor Mobile Robots , 2004, AAAI.

[10]  Eric Fosler-Lussier,et al.  Learning to Temporally Order Medical Events in Clinical Text , 2012, ACL.

[11]  Manfred Pinkal,et al.  Learning Script Knowledge with Web Experiments , 2010, ACL.

[12]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[13]  Gerald DeJong,et al.  Generalizations Based on Explanations , 1981, IJCAI.

[14]  Roni Rosenfeld,et al.  Learning Hidden Markov Model Structure for Information Extraction , 1999 .

[15]  C. Welin.  Scripts, plans, goals and understanding, an inquiry into human knowledge structures: Roger C. Schank and Robert P. Abelson Hillsdale: Lawrence Erlbaum Associates, 1977. 248 pp. £ 10.60 hardcover , 1979 .

[16]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[17]  Nathanael Chambers,et al.  Unsupervised Learning of Narrative Event Chains , 2008, ACL.

[18]  Andreas Stolcke,et al.  Best-first Model Merging for Hidden Markov Model Induction , 1994, ArXiv.

[19]  R. Mooney,et al.  Explanation-Based Learning: An Alternative View , 1986, Machine Learning.

[20]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[21]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.