论文信息 - Relation between Agreement Measures on Human Labeling and Machine Learning Performance : Results from an Art History Image Indexing Domain

Relation between Agreement Measures on Human Labeling and Machine Learning Performance : Results from an Art History Image Indexing Domain

The domain of digital images and texts we focus on parallels the ARTstor Art HistorySurvey Collection (AHSC), a Mellon funded collection of 4,000 images. The AHSC is based on thirteen standard art history survey texts, thus there is a strong correlation between the images and these texts. The AHSC images all have tombstone metadata (e.g., the name of the work, the artist, date, the location of the work), but few have subject matter metadata. We are currently using two of the texts from the AHSC concordance that we scanned and encoded in TEI-Lite (http://www.teic.org/Lite/teiu5 split en.html).

Rebecca J. Passonneau | Tom Lippincott | Tae Yano | Judith Klavans

[1] Claire Grover,et al. Sentence classification experiments for legal text summarisation , 2004 .

[2] Marc Moens,et al. Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status , 2002, CL.

[3] Hsin-Liang Chen,et al. An analysis of image queries in the field of art history , 2001, J. Assoc. Inf. Sci. Technol..

[4] Sara Shatford Layne,et al. Some Issues in the Indexing of Images , 1994, J. Am. Soc. Inf. Sci..

[5] Rebecca J. Passonneau,et al. Measuring Agreement on Set-valued Items (MASI) for Semantic and Pragmatic Annotation , 2006, LREC.

[6] Arlene G. Taylor,et al. Indexing Overlap and Consistency between the "Avery Index to Architectural Periodicals" and the "Architectural Periodicals Index.". , 1993 .

[7] Klaus Krippendorff,et al. Content Analysis: An Introduction to Its Methodology , 1980 .

[8] Nizar Habash,et al. Inter-annotator Agreement on a Multilingual Semantic Annotation Task , 2006, LREC.

[9] Murtha Baca. Practical Issues in Applying Metadata Schemas and Controlled Vocabularies to Cultural Heritage Information , 2003 .

[10] K. Markey. Interindexer consistency tests: a literature review and report of a test of consistency in indexing visual materials , 1984 .

[11] Ron Artstein,et al. = Alpha (or Beta) , 2005 .