How to evaluate the 'goodness' of summaries automatically

................................................................................................................ ii Acknowledgments .................................... ................................................................. iii

[1]  Wai Lam,et al.  Developing Infrastructure for the Evaluation of Single and Multi-document Summarization Systems in a Cross-lingual Environment , 2002, LREC.

[2]  Leonard S. Rutman,et al.  Evaluation Research Methods: A Basic Guide , 1984 .

[3]  Marc Moens,et al.  Argumentative Classification of Extracted Sentences as a First Step Towards Flexible Abstracting , 1999 .

[4]  Kathleen R. McKeown,et al.  Summarization Evaluation Methods: Experiments and Analysis , 1998 .

[5]  Chris D. Paice,et al.  The identification of important concepts in highly structured technical papers , 1993, SIGIR.

[6]  Ellen M. Voorhees,et al.  The Eighth Text REtrieval Conference (TREC-8) , 2000 .

[7]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[8]  I. Dan Melamed,et al.  Automatic Evaluation and Uniform Filter Cascades for Inducing N-Best Translation Lexicons , 1995, VLC@ACL.

[9]  Chris D. Paice,et al.  The automatic generation of literature abstracts: an approach based on the identification of self-indicating phrases , 1980, SIGIR '80.

[10]  James E. Rush,et al.  Automatic abstracting and indexing. II. Production of indicative abstracts by application of contextual inference and syntactic coherence criteria , 1971 .

[11]  Robert L. Donaway,et al.  A Comparison of Rankings Produced by Summarization Evaluation Measures , 2000 .

[12]  Frances C. Johnson,et al.  The application of linguistic processing to automatic abstract generation , 1997 .

[13]  Wai Lam,et al.  Evaluation Challenges in Large-Scale Document Summarization , 2003, ACL.

[14]  Eduard H. Hovy,et al.  Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics , 2003, NAACL.

[15]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[16]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[17]  I. Good.  THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS , 1953 .

[18]  C. Pollard,et al.  Center for the Study of Language and Information , 2022 .

[19]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[20]  Paul Over,et al.  Intrinsic Evaluation of Generic News Text Summarization Systems , 2003 .

[21]  Slava M. Katz,et al.  Estimation of probabilities from sparse data for the language model component of a speech recognizer , 1987, IEEE Trans. Acoust. Speech Signal Process..

[22]  Chin-Yew Lin,et al.  ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation , 2004, COLING.

[23]  D. Sheskin.  Handbook of parametric and nonparametric statistical procedures, 2nd ed. , 2000 .

[24]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[25]  H. P. Edmundson,et al.  New Methods in Automatic Extracting , 1969, JACM.

[26]  R. Weber.  Basic content analysis, 2nd ed. , 1990 .

[27]  Lisa F. Rau,et al.  Information extraction and text summarization using linguistic knowledge acquisition , 1989, Inf. Process. Manag..

[28]  Lisa F. Rau,et al.  Automatic Condensation of Electronic Publications by Sentence Selection , 1995, Inf. Process. Manag..

[29]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[30]  Michael Hoey,et al.  Patterns of Lexis In Text , 1991 .

[31]  Regina Barzilay,et al.  Using Lexical Chains for Text Summarization , 1997 .

[32]  Laurence C. McGinn.  Nonparametric statistics for the behavioral sciences: by Sidney Siegel. 312 pages, 6 × 9 in. New York, McGraw-Hill Book Co., Inc., 1956. Price, $6.50 , 1957 .

[33]  Ellen M. Voorhees,et al.  Variations in relevance judgments and the measurement of retrieval effectiveness , 1998, SIGIR '98.

[34]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[35]  A. Tversky.  Features of Similarity , 1977 .

[36]  Gerard Salton,et al.  Automatic Text Structuring and Summarization , 1997, Inf. Process. Manag..

[37]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[38]  Wai Lam,et al.  Meta-evaluation of Summaries in a Cross-lingual Environment using Content-based Metrics , 2002, COLING.

[39]  John Lyons,et al.  语义学引论 = Linguistic Semantics , 2000 .

[40]  Richard Tucker,et al.  Automatic summarising and the CLASP system , 2000 .

[41]  Chris D. Paice,et al.  Constructing literature abstracts by computer: Techniques and prospects , 1990, Inf. Process. Manag..

[42]  Joseph P. Turian,et al.  Evaluation of machine translation and its evaluation , 2003, MTSUMMIT.

[43]  Mohamed Benbrahim,et al.  Automatic text summarisation through lexical cohesion analysis , 1996 .

[44]  Peter Hughes,et al.  Content Analysis: An Introduction to Its Methodology [Book Review] , 2004 .

[45]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[46]  E. F. Skorochod'ko.  Adaptive Method of Automatic Abstracting and Indexing , 1971, IFIP Congress.

[47]  Kathleen F. McCoy,et al.  Efficient text summarization using lexical chains , 2000, IUI '00.

[48]  Jerry R. Hobbs.  Literature And Cognition , 1990 .

[49]  Seiji Miike,et al.  A full-text retrieval system with a dynamic abstract generation function , 1994, SIGIR '94.

[50]  F ChenStanley,et al.  An Empirical Study of Smoothing Techniques for Language Modeling , 1996, ACL.

[51]  Antonio Zamora,et al.  Automatic Abstracting Research at Chemical Abstracts Service , 1975, J. Chem. Inf. Comput. Sci..

[52]  Khurshid Ahmad,et al.  Choosing Feature Sets for Training and Testing Self-Organising Maps: A Case Study , 2001, Neural Computing & Applications.

[53]  Michael Halliday,et al.  Cohesion in English , 1976 .

[54]  Khurshid Ahmad,et al.  Summary evaluation and text categorization , 2003, SIGIR '03.

[55]  Ellen M. Voorhees,et al.  Overview of the TREC-9 Question Answering Track , 2000, TREC.

[56]  Lynette Hirschman,et al.  Evaluating Message Understanding Systems: An Analysis of the Third Message Understanding Conference (MUC-3) , 1993, CL.

[57]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[58]  George R. Doddington,et al.  Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics , 2002 .

[59]  Jade Goldstein-Stewart,et al.  Summarizing text documents: sentence selection and evaluation metrics , 1999, SIGIR '99.

[60]  Dragomir R. Radev,et al.  Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies , 2000, ArXiv.

[61]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[62]  Karen Spärck Jones.  Automatic language and information processing: rethinking evaluation , 2001, Natural Language Engineering.

[63]  G Salton,et al.  Automatic Analysis, Theme Generation, and Summarization of Machine-Readable Texts , 1994, Science.

[64]  Therese Firmin Hand,et al.  A Proposal for Task-based Evaluation of Text Summarization Systems , 1997, Workshop On Intelligent Scalable Text Summarization.

[65]  Mark T. Maybury,et al.  Automatic Summarization , 2002, Computational Linguistics.

[66]  Ian H. Witten,et al.  The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression , 1991, IEEE Trans. Inf. Theory.