Natural language neural network and its application to question-answering system

This paper proposes a novel neural network to treat natural language. Most of the conventional neural networks can only process sentences consisted of a few words, and their applications are very simple such as metaphor understanding. The proposed network can process many complicated sentences and can be used as an associative memory and a question-answering system. The proposed network is composed of 3 layers and one network: Sentence Layer, Knowledge Layer, Deep Case Layer and Dictionary Network. The input sentences are divided into knowledge units and stored in the Knowledge Layer. The Deep Case Layer play an important role to process the knowledge units properly. The Dictionary Network also plays an important role as a knowledge based. We have carried out several experiments and they have shown that the proposed neural network has superior performances as an associative memory and a question-answering system. Especially as a question-answering system, the performance is very close to the elaborated system based on artificial intelligence.

[1]  Yuji Matsumoto,et al.  Japanese Dependency Analysis using Cascaded Chunking , 2002, CoNLL.

[2]  B Opitz,et al.  Distributed cortical networks for syntax processing: Broca’s area as the common denominator , 2003, Brain and Language.

[3]  Lubica Benuskova,et al.  Mapping sensorimotor sequences to word sequences: A connectionist model of language acquisition and sentence generation , 2012, Cognition.

[4]  Madoka Ishioroshi,et al.  Query expansion method using answer candidates and the effect of combining their results on Web question-answering , 2009 .

[5]  Iraklis Varlamis,et al.  Text Relatedness Based on a Word Thesaurus , 2010, J. Artif. Intell. Res..

[6]  Noriaki Yahata,et al.  Selective enhancement of functional connectivity in the left prefrontal cortex during sentence processing , 2003, NeuroImage.

[7]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[8]  Jimmy J. Lin,et al.  Overview of the TREC 2007 Question Answering Track , 2008, TREC.

[9]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[10]  Ben Goertzel,et al.  A world survey of artificial brain projects, Part I: Large-scale brain simulations , 2010, Neurocomputing.

[11]  原田 実,et al.  Answering improvement of QA system Metis based on semantic graph matching - Focused on question analysis and knowledge retrieval - (in Japanese) , 2009 .

[12]  Yuji Matsumoto,et al.  Applying Conditional Random Fields to Japanese Morphological Analysis , 2004, EMNLP.

[13]  Masafumi Hagiwara,et al.  Natural language processing neural network for analogical inference , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[14]  Masafumi Hagiwara,et al.  Natural language processing neural network considering deep cases , 2011 .

[15]  Masafumi Hagiwara,et al.  Natural language neural network and its application to question-answering system , 2012, IJCNN.

[16]  K. Holyoak,et al.  A symbolic-connectionist theory of relational inference and generalization. , 2003, Psychological review.

[17]  James L. McClelland,et al.  Generalization Through the Recurrent Interaction of Episodic Memories , 2012, Psychological review.

[18]  A. Glenberg,et al.  Symbol Grounding and Meaning: A Comparison of High-Dimensional and Embodied Theories of Meaning , 2000 .

[19]  Masafumi Hagiwara,et al.  Word vectorization using relations among words for neural network , 2010 .

[20]  K. Holyoak,et al.  A neurocomputational system for relational reasoning , 2012, Trends in Cognitive Sciences.

[21]  Shun Ishizaki,et al.  A Geometric Method of Extracting Salient Features for Metaphor Understanding , 2000 .

[22]  Iraklis Varlamis,et al.  Word Sense Disambiguation with Semantic Networks , 2008, TSD.

[23]  Rui Liu,et al.  Sentence generation for artificial brains: A glocal similarity-matching approach , 2010, Neurocomputing.

[24]  Günther Palm,et al.  Neural associative memories for the integration of language, vision and action in an autonomous agent , 2009, Neural Networks.

[25]  Tsuneaki Kato,et al.  Question Answering Challenge for Five Ranked Answers and List Answers - Overview of NTCIR4 QAC2 Subtask 1 and 2 , 2004, NTCIR.

[26]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[27]  Ben Goertzel,et al.  A world survey of artificial brain projects, Part II: Biologically inspired cognitive architectures , 2010, Neurocomputing.