Combining Medical Domain Ontological Knowledge and Low-level Image Features for Multimedia Indexing

Biomedical images are invaluable in establishing diagnosis, acquiring technical skills, and implementing best practices in many areas of medicine. At present, images needed for instructional purposes or in support of clinical decisions appear in specialized databases and in biomedical articles, and are therefore not easily accessible. Our goal is to automatically annotate images extracted from scientific publications with respect to their usefulness for clinical decision support and instructional purposes, and project the annotations onto images stored in databases by linking images through content-based image similarity. This paper presents an overview of our approach to automatic image indexing, content-based image analysis, and the results of a pilot evaluation of an automatic indexing method based on biomedical terms extracted from snippets of text pertaining to images appearing in scientific biomedical articles.