Exploring rich evidence for maximum entropy-based question answering

Open domain automated Question Answering (QA) aims to automatically answer users'; questions in spoken language. I propose a maximum entropy-based ranking model which effectively integrates various features, including orthographic, lexical, surface pattern, syntactic and semantic features for the answer extraction. To effectively capture syntactic evidence, I present two methods: dependency relation pattern methods and dependency relation path correlation method. Both methods overcome the problems arising from the divergences of lexical representations between question and answer sentences. I experimentally demonstrate that both methods greatly outperform the state-of-the-art syntactic answer extraction methods on TREC datasets. To capture semantic evidence, I propose an automatic method to incorporate FrameNet-style semantic role information. The graph-theoretic framework goes some way towards addressing coverage problems related with FrameNet and formulates the similarity measure of semantic structures as a graph matching problem. Experimental results show that the FrameNet-based semantic features may further boost the performance on the answer extraction module. Furthermore, I propose a maximum entropy-based ranking model to incorporate all captured information. As a result, the model using the optimal feature combination achieves top-ranked performance among all of the participants world-wide in the most recent TREC evaluation. Domanen-unabhangige automatische Frage-Antwort (QA) is zur automatische Antwort auf die Fragen der Benutzer in muendliche Sprache. Ich stelle eine maximal Entropie-basisbezogen Ranking Modul auf, das tatsachlich integriert verschiedene Features, inkl. orthographisches, lexikalisches, Oberflache Muster, syntaktisches und semantisches Features fuer die Antwort Extraktion. Um eine tatsachliche Erfassung der syntaktische Beweise zu erhalten, ich reprasentiere zwei Methoden: Abhangigkeit Beziehung Muster und Abhangigheit Beziehung Pfad Zusammenhang. Ich demonstriere versuchsweise, dass die beide Methoden die modernste syntaktische Antwort Extraktion Methoden on TREC Datensatz uebertrifft. Um die semantische Beweise zu erfassen, ich stelle eine automatische Methode auf, dadurch wird semantische Rolle Information in FrameNet-Art inkorporiert. Das experimentell Ergebnis dass die FrameNet-basisbezogene semantische Features die Leistung on Antwort Extraktion Modul. Darueber hinaus stelle ich einen maximal Entropie-basisbezogenen Ranking Modul, um alle erfasste Information zu inkorporieren. Als Ergebnis, der Modul, der die optimale Feature Kombination benutzt, erreicht top-ranked Leistung unter alle Teilnehmer weltweit in letzte TREC Bewertung.

[1]  Sanda M. Harabagiu,et al.  Temporal Context Representation and Reasoning , 2005, IJCAI.

[2]  Scott Miller,et al.  Answer Selection and Confidence Estimation , 2003, New Directions in Question Answering.

[3]  Andreas Merkel,et al.  Language Model Based Query Classification , 2007, ECIR.

[4]  Hoa Trang Dang,et al.  Overview of the TREC 2006 Question Answering Track 99 , 2006, TREC.

[5]  Grace Hui Yang,et al.  The Integration of Lexical Knowledge and External Resources for Question Answering , 2002, TREC.

[6]  Adwait Ratnaparkhi,et al.  A Maximum Entropy Approach to Identifying Sentence Boundaries , 1997, ANLP.

[7]  Jinxi Xu,et al.  TREC 2003 QA at BBN: Answering Definitional Questions , 2003, TREC.

[8]  Dietrich Klakow,et al.  Exploring Correlation of Dependency Relation Paths for Answer Extraction , 2006, ACL.

[9]  Andreas Merkel,et al.  Improved methods for language model based question classification , 2007, INTERSPEECH.

[10]  Mirella Lapata,et al.  Optimal Constituent Alignment with Edge Covers for Semantic Projection , 2006, ACL.

[11]  Dan Moldovan,et al.  Using Knowledge Extraction and Maintenance Techniques To Enhance Analytical Performance , 2005 .

[12]  Mats Rooth,et al.  Looking Under the Hood : Tools for Diagnosing your Question Answering Engine , 2001, ACL 2001.

[13]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[14]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Bonnie L. Webber,et al.  Experiments at the University of Edinburgh for the TREC 2006 QA Track , 2006, TREC.

[16]  A. Volgenant,et al.  A shortest augmenting path algorithm for dense and sparse linear assignment problems , 1987, Computing.

[17]  Ben Taskar,et al.  A Discriminative Matching Approach to Word Alignment , 2005, HLT.

[18]  Daniel Jurafsky,et al.  Shallow Semantic Parsing using Support Vector Machines , 2004, NAACL.

[19]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[20]  Sanda M. Harabagiu,et al.  LCC Tools for Question Answering , 2002, TREC.

[21]  Adwait Ratnaparkhi,et al.  Learning to Parse Natural Language with Maximum Entropy Models , 1999, Machine Learning.

[22]  Charles L. A. Clarke,et al.  Exploiting redundancy in question answering , 2001, SIGIR '01.

[23]  Teruko Mitamura,et al.  Language-independent Probabilistic Answer Ranking for Question Answering , 2007, ACL.

[24]  Dan I. Moldovan,et al.  Lexical Chains for Question Answering , 2002, COLING.

[25]  Bernardo Magnini,et al.  Combining Linguistic Processing and Web Mining for Question Answering: ITC-irst at TREC 2004 , 2004, TREC.

[26]  Eduard H. Hovy,et al.  Learning surface text patterns for a Question Answering System , 2002, ACL.

[27]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[28]  Daniel Marcu,et al.  A Noisy-Channel Approach to Question Answering , 2003, ACL.

[29]  Adwait Ratnaparkhi,et al.  IBM's Statistical Question Answering System , 2000, TREC.

[30]  Adwait Ratnaparkhi,et al.  Question Answering Using Maximum-Entropy Components , 2001, NAACL.

[31]  A. Merkel,et al.  3 Comparing Improved Language Models for Sentence Retrieval in Question Answering , 2007 .

[32]  Tat-Seng Chua,et al.  National University of Singapore at the TREC 13 Question Answering Main Task , 2004, TREC.

[33]  Christopher R. Johnson,et al.  Background to Framenet , 2003 .

[34]  Ganesh Ramakrishnan,et al.  Passage Scoring for Question Answering via Bayesian Inference on Lexical Relations , 2003, TREC.

[35]  Hiroaki Sakoe,et al.  A Dynamic Programming Approach to Continuous Speech Recognition , 1971 .

[36]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[37]  Vasile Rus,et al.  Logic Form Transformation of WordNet and its Applicability to Question Answering , 2001, ACL.

[38]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[39]  Charles L. A. Clarke,et al.  Web Reinforced Question Answering (MultiTest Experiments for TREC 2001) , 2001, TREC.

[40]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[41]  Jimmy J. Lin.  The Web as a Resource for Question Answering: Perspectives and Challenges , 2002, LREC.

[42]  Eduard Hovy,et al.  Statistical QA - Classifier vs. Re-ranker: What’s the difference? , 2003, ACL 2003.

[43]  李幼升,et al.  Ph , 1989 .

[44]  Grace Hui Yang,et al.  QUALIFIER In TREC-12 QA Main Task , 2003, TREC.

[45]  Dekang Lin,et al.  PRINCIPAR - An Efficient, Broad-coverage, Principle-based Parser , 1994, COLING.

[46]  Sanda M. Harabagiu,et al.  Answering Complex, List and Context Questions with LCC's Question-Answering Server , 2001, TREC.

[47]  Heikki Mannila,et al.  Distance measures for point sets and their computation , 1997, Acta Informatica.

[48]  Ellen M. Voorhees,et al.  Overview of the TREC-9 Question Answering Track , 2000, TREC.

[49]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[50]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[51]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[52]  Andreas Merkel,et al.  Dedicated Backing-Off Distributions for Language Model Based Passage , 2006, LWA.

[53]  Tilman Becker,et al.  Question Answering by Searching Large Corpora With Linguistic Methods , 2004, TREC.

[54]  Johan Bos.  The "La Sapienza" Question Answering System at TREC 2006 , 2006, TREC.

[55]  Tat-Seng Chua,et al.  Question answering passage retrieval using dependency relations , 2005, SIGIR '05.

[56]  Sanda M. Harabagiu,et al.  Employing Two Question Answering Systems in TREC 2005 , 2005, TREC.

[57]  S. Levinson,et al.  Considerations in dynamic time warping algorithms for discrete word recognition , 1978 .

[58]  Dell Zhang,et al.  Web Based Pattern Mining and Matching Approach to Question Answering , 2002, TREC.

[59]  Jimmy J. Lin,et al.  Web question answering: is more always better? , 2002, SIGIR '02.

[60]  Daniel Marcu,et al.  Multiple-Engine Question Answering in TextMap , 2003, TREC.

[61]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Report , 1999, TREC.

[62]  Ronald Rosenfeld,et al.  Adaptive Statistical Language Modeling; A Maximum Entropy Approach , 1994 .

[63]  Sanda M. Harabagiu,et al.  COGEX: A Logic Prover for Question Answering , 2003, NAACL.

[64]  Martin M. Soubbotin.  Patterns of Potential Answer Expressions as Clues to the Right Answers , 2001, TREC.

[65]  Sanda M. Harabagiu,et al.  Answer Mining by Combining Extraction Techniques with Abductive Reasoning , 2003, Text Retrieval Conference.

[66]  Ralph Grishman,et al.  Exploiting Diverse Knowledge Sources via Maximum Entropy in Named Entity Recognition , 1998, VLC@COLING/ACL.

[67]  Sanda M. Harabagiu,et al.  Question Answering Based on Semantic Structures , 2004, COLING.

[68]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[69]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[70]  Steven Abney,et al.  Parsing By Chunks , 1991 .

[71]  M. de Rijke,et al.  Tequesta: The University of Amsterdam's Textual Question Answering System , 2001, TREC.

[72]  Stanley F. Chen,et al.  A Gaussian Prior for Smoothing Maximum Entropy Models , 1999 .

[73]  G. W. Hughes,et al.  Minimum Prediction Residual Principle Applied to Speech Recognition , 1975 .

[74]  Mitchell P. Marcus,et al.  Maximum entropy models for natural language ambiguity resolution , 1998 .

[75]  Noah A. Smith,et al.  What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA , 2007, EMNLP.

[76]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[77]  Rob Koeling.  Chunking with Maximum Entropy Models , 2000, CoNLL/LLL.

[78]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[79]  Xuanjing Huang,et al.  FDUQA on TREC2003 QA task , 2003, TREC.