A Bayesian-Network Approach to Lexical Disambiguation

Lexical ambiguity can be syntactic if it involves more than one grammatical category for a single word, or semantic if more than one meaning can be associated with a word. In this article we discuss the application of a Bayesian-network model in the resolution of lexical ambiguities of both types. The network we propose comprises a parsing subnetwork, which can be constructed automatically for any context-free grammar, and a subnetwork for semantic analysis, which, in the spirit of Fillmore's (1968) case grammars, seeks to fulfill the required cases of all candidates for verb of the sentence. Solving for the highest joint probability of the variables conditioned upon the evidences to the network yields the most likely candidate with its meaning, along with its cases and respective meanings. This is achieved by fixing the values of all evidence nodes concurrently, and then performing a stochastic simulation in which the remaining nodes are updated probabilistically with a high degree of parallelism. The process of disambiguation is directed neither by the syntax nor the semantics, but rather by the interrelation between the two subnetworks. The use of a Bayesian-network model allows us to express this interrelation between the two subnetworks and among their constituents in a rather direct and rigorous way that, in connection with the convergence properties of the stochastic simulation, reveals a very robust model.

[1]  Garrison W. Cottrell,et al.  A connectionist approach to word sense disambiguation , 1988 .

[2]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[3]  Fahiem Bacchus,et al.  Representing and reasoning with probabilistic knowledge , 1988 .

[4]  Max Henrion,et al.  An Introduction to Algorithms for Inference in Belief Nets , 1989, UAI.

[5]  Tomas Hrycej,et al.  Gibbs Sampling in Bayesian Networks , 1990, Artif. Intell..

[6]  Terry Winograd,et al.  Language as a Cognitive Process , 1983, CL.

[7]  Steven L. Lytinen.  Semantics-First Natural Language Processing , 1991, AAAI.

[8]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[9]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[10]  Michael Goldsmith,et al.  Programming in occam 2 , 1985, Prentice Hall international series in computer science.

[11]  Robert P. Goldman,et al.  A Probabilistic Model of Plan Recognition , 1991, AAAI.

[12]  Curt Burgess,et al.  Implications of Lexical Ambiguity Resolution for Word Recognition and Comprehension , 1988 .

[13]  Valmir Carneiro Barbosa,et al.  On the distributed parallel simulation of Hopfield's neural networks , 1990, Softw. Pract. Exp..

[14]  Graeme Hirst,et al.  Race-Based Parsing and Syntactic Disambiguation , 1990, Cogn. Sci..

[15]  James F. Allen.  Natural language understanding , 1987, Bejnamin/Cummings series in computer science.

[16]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[17]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[18]  Mark A. Fanty,et al.  Context-free parsing with connectionist networks , 1987 .

[19]  Eugene Charniak,et al.  Passing Markers: A Theory of Contextual Influence in Language Comprehension* , 1983 .

[20]  Peter C. Cheeseman,et al.  In Defense of Probability , 1985, IJCAI.

[21]  Fahiem Bacchus,et al.  Representing and reasoning with probabilistic knowledge - a logical approach to probabilities , 1991 .

[22]  Robert P. Goldman,et al.  A Semantics for Probabilistic Quantifier-Free First-Order Languages, with Particular Application to Story Understanding , 1989, IJCAI.

[23]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[24]  Robert P. Goldman,et al.  Probabilistic text understanding , 1992 .

[25]  Eugene Charniak,et al.  Passing Markers: A Theory of Contextual Influence in Language Comprehension , 1983, Cogn. Sci..

[26]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[27]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[29]  Peter Norvig,et al.  A Critical Evaluation of Commensurable Abduction Models for Semantic Interpretation , 1990, COLING.

[30]  Eli Gafni,et al.  A Distributed Implementation of Simulated Annealing , 1989, J. Parallel Distributed Comput..