Biologically Plausible Connectionist Prediction of Natural Language Thematic Relations

In Natural Language Processing (NLP) symbolic systems, several linguistic phenomena, for instance, the thematic role relationships between sentence constituents, such as agent, patient, and location, can be accounted for by the employment of a rule-based grammar. Another approach to NLP concerns the use of the connectionist model, which has the benefits of learning, generalization and fault tolerance, among others. A third option merges the two previous approaches into a hybrid one: a symbolic thematic theory is used to supply the connectionist network with initial knowledge. Inspired on neuroscience, it is proposed a symbolic-connectionist hybrid system called BioθPred (Biologically plausible thematic (θ) symbolic-connectionist Predictor), designed to reveal the thematic grid assigned to a sentence. Its connectionist architecture comprises, as input, a featural representation of the words (based on the verb/noun WordNet classification and on the classical semantic microfeature representation), and, as output, the thematic grid assigned to the sentence. BioθPred is designed to “predict” thematic (semantic) roles assigned to words in a sentence context, employing biologically inspired training algorithm and architecture, and adopting a psycholinguistic view of thematic theory.

[1]  A. Alishahi,et al.  Please Scroll down for Article Language and Cognitive Processes a Computational Model of Learning Semantic Roles from Child-directed Language a Computational Model of Learning Semantic Roles from Child-directed Language , 2022 .

[2]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[3]  Randall C. O'Reilly,et al.  Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm , 1996, Neural Computation.

[4]  Neville Ryant,et al.  A large-scale classification of English verbs , 2008, Lang. Resour. Evaluation.

[5]  Todd R. Ferretti,et al.  Thematic Roles as Verb-specific Concepts , 1997 .

[6]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[7]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[8]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  Joydeep Ghosh,et al.  Symbolic Interpretation of Artificial Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[11]  David R. Dowty.  Thematic proto-roles and argument selection , 1991 .

[12]  Alexander H. Waibel,et al.  Incremental Parsing by Modular Recurrent Connectionist Networks , 1989, NIPS.

[13]  C. Lee Giles,et al.  Rule Revision With Recurrent Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[14]  João Luís Garcia Rosa,et al.  Linguistic Relations Encoding in a Symbolic-Connectionist Hybrid Natural Language Processor , 2000, IBERAMIA-SBIA.

[15]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[16]  Janyce Wiebe,et al.  Empirical acquisition of conceptual distinctions via dictionary definitions , 2005 .

[17]  Jude W. Shavlik,et al.  Combining Symbolic and Neural Learning , 1994, Machine Learning.

[18]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[19]  Geoffrey E. Hinton,et al.  Learning Representations by Recirculation , 1987, NIPS.

[20]  Tim van Gelder,et al.  Defining ‘Distributed Representation’ , 1992 .

[21]  João Luís Garcia Rosa,et al.  Biologically plausible connectionist prediction of natural language thematic relations , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[22]  Ajay N. Jain,et al.  Parsing Complex Sentences with Structured Connectionist Networks , 1991, Neural Computation.

[23]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[24]  Peter Ford Dominey,et al.  Neural network processing of natural language: II. Towards a unified model of corticostriatal function in learning sentence comprehension and non-linguistic sequencing , 2009, Brain and Language.

[25]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[26]  Li-Min Fu,et al.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[27]  Francis Crick,et al.  The recent excitement about neural networks , 1989, Nature.

[28]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[29]  Sandiway Fong,et al.  Natural Language Grammatical Inference with Recurrent Neural Networks , 2000, IEEE Trans. Knowl. Data Eng..

[30]  João Luís Garcia Rosa,et al.  A biologically inspired connectionist system for natural language processing , 2002, VII Brazilian Symposium on Neural Networks, 2002. SBRN 2002. Proceedings..

[31]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[32]  J. Shavlik,et al.  The Extraction of Reened Rules from Knowledge-based Neural Networks , 1993 .

[33]  João Luís Garcia Rosa.  A Biologically Motivated and Computationally Efficient Natural Language Processor , 2004, MICAI.

[34]  João Luís Garcia Rosa,et al.  A Connectionist Thematic Grid Predictor for Pre-parsed Natural Language Sentences , 2007, ISNN.

[35]  Douglas L. T. Rohde,et al.  Language acquisition in the absence of explicit negative evidence: how important is starting small? , 1999, Cognition.

[36]  Geoffrey E. Hinton.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1991 .

[37]  Geoffrey E. Hinton,et al.  Proceedings of the 1988 Connectionist Models Summer School , 1989 .

[38]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[39]  Christiane Fellbaum,et al.  English Verbs as a Semantic Net , 1990 .

[40]  Risto Miikkulainen,et al.  Natural Language Processing With Modular PDP Networks and Distributed Lexicon , 1991, Cogn. Sci..

[41]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[42]  James L. McClelland,et al.  Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences , 1986 .

[43]  João Luís Garcia Rosa,et al.  Hybrid Thematic Role Processor: Symbolic Linguistic Relations Revised by Connectionist Learning , 1999, IJCAI.

[44]  A. Andrews.  The major functions of the noun phrase , 2007 .

[45]  David R. Dowty.  On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[46]  John F. Sowa,et al.  Knowledge representation: logical, philosophical, and computational foundations , 2000 .

[47]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[48]  Vasant Honavar,et al.  Integrating Symbol Processing Systems and Connectionist Networks , 1995 .

[49]  João Luís Garcia Rosa,et al.  A HYBrid symbolic-connectionist processor of natural language semantic relations , 2009, 2009 IEEE Workshop on Hybrid Intelligent Models and Applications.

[50]  J.L.G. Rosa,et al.  Thematic role assignment through a biologically plausible symbolic-connectionist hybrid system , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[51]  George A. Miller,et al.  Nouns in WordNet: A Lexical Inheritance System , 1990 .

[52]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.