RadSem: Semantic Annotation and Retrieval for Medical Images

We present a tool for semantic medical image annotation and retrieval. It leverages the MEDICO ontology which covers formal background information from various biomedical ontologies such as the Foundational Model of Anatomy (FMA), terminologies like ICD-10 and RadLex and covers various aspects of clinical procedures. This ontology is used during several steps of annotation and retrieval: (1) We developed an ontology-driven metadata extractor for the medical image format DICOM. Its output contains, e. g. , person name, age, image acquisition parameters, body region, etc . (2) The output from (1) is used to simplify the manual annotation by providing intuitive visualizations and to provide a preselected subset of annotation concepts. Furthermore, the extracted metadata is linked together with anatomical annotations and clinical findings to generate a unified view of a patient's medical history. (3) On the search side we perform query expansion based on the structure of the medical ontologies. (4) Our ontology for clinical data management allows us to link and combine patients, medical images and annotations together in a comprehensive result list. (5) The medical annotations are further extended by links to external sources like Wikipedia to provide additional information.

[1]  Dean Allemang,et al.  The Semantic Web - ISWC 2006, 5th International Semantic Web Conference, ISWC 2006, Athens, GA, USA, November 5-9, 2006, Proceedings , 2006, SEMWEB.

[2]  Michael Kohnen,et al.  Quality of DICOM header information for image categorization , 2002, SPIE Medical Imaging.

[3]  Mohd Sapiyan Baba,et al.  Automatic Multilevel Medical Image Annotation and Retrieval , 2008, Journal of Digital Imaging.

[4]  Manuel Möller,et al.  A Scalable Architecture for Cross-Modal Semantic Annotation and Retrieval , 2008, KI.

[5]  Gilles Kassel,et al.  Towards an ontology for sharing medical images and regions of interest in neuroimaging , 2008, J. Biomed. Informatics.

[6]  Paul Buitelaar,et al.  Medical Image Understanding Through the Integration of Cross-Modal Object Recognition with Formal Domain Knowledge , 2008, HEALTHINF.

[7]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[8]  C. Langlotz.  RadLex: a new method for indexing online educational materials. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[9]  Jacob D. Furst,et al.  Modelling semantics from image data: opportunities from LIDC , 2010 .

[10]  Andreas Dengel,et al.  DynaQ - Dynamic Queries for Electronic Document Management , 2006, 2006 10th IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW'06).

[11]  Aleksandra Mojsilovic,et al.  Semantic-Friendly Indexing and Quering of Images Based on the Extraction of the Objective Semantic Cues , 2004, International Journal of Computer Vision.

[12]  Deborah L. McGuinness,et al.  OWL Web ontology language overview , 2004 .

[13]  Raphael Volz,et al.  Cooking the Semantic Web with the OWL API , 2003, SEMWEB.

[14]  Frank Bomarius,et al.  KI 2008: Advances in Artificial Intelligence, 31st Annual German Conference on AI, KI 2008, Kaiserslautern, Germany, September 23-26, 2008. Proceedings , 2008, KI.

[15]  Cornelius Rosse,et al.  A Reference Ontology for Bioinformatics: The Foundational Model of Anatomy , 2003 .

[16]  John Mylopoulos,et al.  The Semantic Web - ISWC 2003 , 2003, Lecture Notes in Computer Science.

[17]  Daniel L. Rubin,et al.  FMA-RadLex: An Application Ontology of Radiological Anatomy derived from the Foundational Model of Anatomy Reference Ontology , 2008, AMIA.

[18]  José L. V. Mejino,et al.  A reference ontology for biomedical informatics: the Foundational Model of Anatomy , 2003, J. Biomed. Informatics.

[19]  David R. Karger,et al.  Fresnel: A Browser-Independent Presentation Vocabulary for RDF , 2005, SEMWEB.

[20]  Mary K Pulvermacher,et al.  Toward the Use of an Upper Ontology for U.S. Government and U.S. Military Domains: An Evaluation , 2004 .

[21]  Daniel L. Rubin,et al.  Medical Imaging on the Semantic Web: Annotation and Image Markup , 2008, AAAI Spring Symposium: Semantic Scientific Knowledge Integration.

[22]  Zhongfei Zhang,et al.  Automatic Medical Image Annotation and Retrieval using SEMI-SECC , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[23]  Zhongfei Zhang,et al.  Automatic Medical Image Annotation and Retrieval Using SECC , 2006, 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).

[24]  O Bodenreider,et al.  Biomedical ontologies in action: role in knowledge management, data integration and decision support. , 2008, Yearbook of medical informatics.