The IRMA Project: A State of the Art Report on Content-Based Image Retrieval in Medical Applications

The objective of this work is to develop a general structure for semantic image analysis that is suitable for content-based image retrieval in medical applications and an architecture for its efficient implementation. Stepwise content analysis of medical images results in six layers of information modeling (raw data layer, registered data layer, feature layer, scheme layer, object layer, knowledge layer). Medical expert knowledge is incorporated into several layers. In the registered data layer, a reference database with 10,000 images categorized according to the image modality, orientation, body region examined, and biological system imaged is used. By means of prototypes in each category, identification of objects and their geometrical or temporal relationships are handled in the object and the knowledge layer, respectively. Depending on the complexity of the query, it is processed on the higher layers starting with the scheme layer, where a hierarchical blob representation of image content is provided. Here, local image similarity is assessed by graph matching. The multilayer processing is implemented using a distributed system designed with only three core elements: (i) the central database holds program sources, processing scheme descriptions, images, features, blob trees, and administrative information about the workstation cluster; (ii) the scheduler balances distributed computing by addressing daemons running on all connected workstations; and (iii) the web server provides graphical user interfaces for data entry and retrieval, which can be easily adapted to a variety of applications for content-based image retrieval in medicine. Since manual labeling of reference data is still in progress, the system was used so far for processing primitive queries, i.e. queries regarding the category. However, since all feature transformations in all semantic layers are based the same implemented mechanism, this is sufficient to validate the overall system concept. The leaving-oneout experiments were distributed by the scheduler and controlled via corresponding job lists. The experiments have shown that the IRMA framework offers transparency regarding the viewpoint of a distributed system and the user, such as (i) location and access transparency for data and program sources; (ii) replication transparency for programs in development; (iii) concurrency transparency for job processing and feature extraction; (iv) system transparency at method implementation time; and (v) job distribution transparency when issuing a query. The proposed architecture is suitable for content-based image retrieval in medical applications. It improves current picture archiving and communication systems that still rely on alphanumerical descriptions, which are insufficient for image retrieval of high recall and precision.

[1]  Christos Faloutsos,et al.  Fast and Effective Retrieval of Medical Tumor Shapes , 1998, IEEE Trans. Knowl. Data Eng..

[2]  James Ze Wang,et al.  Content-based image indexing and searching using Daubechies' wavelets , 1998, International Journal on Digital Libraries.

[3]  Thomas Martin Deserno,et al.  Platform for distributed image processing and image retrieval , 2003, Visual Communications and Image Processing.

[4]  Sameer Antani,et al.  Biomedical information from a national collection of spine x-rays: film to content-based retrieval , 2003, SPIE Medical Imaging.

[5]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Electronic Imaging.

[6]  Thomas Martin Deserno,et al.  Hierarchical feature clustering for content-based retrieval in medical image databases , 2003, SPIE Medical Imaging.

[7]  James S. Duncan,et al.  Synthesis of Research: Medical Image Databases: A Content-based Retrieval Approach , 1997, J. Am. Medical Informatics Assoc..

[8]  Edward A. Lee,et al.  Dataflow process networks , 2001 .

[9]  Thomas Martin Deserno,et al.  Automatic detection of the view position of chest radiographs , 2003, SPIE Medical Imaging.

[10]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[11]  Erhard Rahm,et al.  Similarity flooding: a versatile graph matching algorithm and its application to schema matching , 2002, Proceedings 18th International Conference on Data Engineering.

[12]  Babu M. Mehtre,et al.  CORE: a content-based retrieval engine for multimedia information systems , 1995, Multimedia Systems.

[13]  Chong-Wah Ngo,et al.  Exploiting image indexing techniques in DCT domain , 2001, Pattern Recognit..

[14]  Bidyut Baran Chaudhuri,et al.  Texture Segmentation Using Fractal Dimension , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Thomas M. Lehmanna,et al.  Header for SPIE use Content-Based Image Retrieval in Medical Applications: A Novel Multi-Step Approach , 2022 .

[16]  Yann LeCun,et al.  Efficient Pattern Recognition Using a New Transformation Distance , 1992, NIPS.

[17]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[18]  Thomas Martin Deserno,et al.  Content-based image retrieval in medical applications: a novel multistep approach , 1999, Electronic Imaging.

[19]  Edward A. Lee,et al.  Dataflow process networks , 1995, Proc. IEEE.

[20]  J. T. Robinson,et al.  Progressive search and retrieval in large image archives , 1998, IBM J. Res. Dev..

[21]  Ricky K. Taira,et al.  Knowledge-Based Image Retrieval with Spatial and Temporal Constructs , 1998, IEEE Trans. Knowl. Data Eng..

[22]  Hermann Ney,et al.  Classification of Medical Images Using local Representations , 2002, Bildverarbeitung für die Medizin.

[23]  Michael Kohnen,et al.  The IRMA code for unique classification of medical images , 2003, SPIE Medical Imaging.

[24]  Kristina Schädler,et al.  Comparing Structures Using a Hopfield-Style Neural Network , 1999, Applied Intelligence.

[25]  Michele Nappi,et al.  FIRST: Fractal Indexing and Retrieval SysTem for Image Databases , 1998, Image Vis. Comput..

[26]  Jitendra Malik,et al.  Color- and texture-based image segmentation using EM and its application to content-based image retrieval , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[27]  Thomas S. Huang,et al.  Edge-based structural features for content-based image retrieval , 2001, Pattern Recognit. Lett..

[28]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Mohan M. Trivedi,et al.  Graph matching using a direct classification of node attendance , 1996, Pattern Recognit..

[30]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Carla E. Brodley,et al.  ASSERT: A Physician-in-the-Loop Content-Based Retrieval System for HRCT Image Databases , 1999, Comput. Vis. Image Underst..

[32]  Hermann Ney,et al.  Statistical framework for model-based image retrieval in medical applications , 2003, J. Electronic Imaging.