Multiscale salient point-based retrieval of fracture cases

Fractures are common injuries, some complicated fractures may require a surgical intervention. When such an operation is planned it can be beneficial to have access to similar past cases including follow ups to compare, which method might be the most adapted one in a particular situation. At the orthopaedic service of the University hospitals of Geneva a database of past cases including pre- and post-operative images and case descriptions has been created over the past years with the goal to support clinical decision making. Images play an important role in the decision making process and the judgment of a fracture, but visual image content is currently not directly accessible for search. At the moment, search is mainly via a classification system of the fractures or in the patient record itself only by patient ID. In this paper we propose a solution that combines visual information from several images in a case to calculate similarity between cases and allow thus an access to visually similar cases. Such a system can complement the text- or classification-based search that has been used so far. In a preliminary study, we used pixel-grid-based salient-point features to build a first prototype of case-based visual retrieval of fracture cases. Cases belonging to different fracture classes were beforehand often confused due to the similar bone structures in the various images. In this article, a multi-scale approach is used in order to perform similarity measures at both large and small scales. When compared to the first prototype, the introduction of scale and spatial information allowed improving the performance of the system. Cases containing similar bone structures but with dissimilar fractures are generally ranked lower whereas more relevant cases are returned. The system can thus be expected to perform sufficiently well for use in clinical practice and particularly for teaching.

[1]  T M Lehmann,et al.  Content-based Image Retrieval in Medical Applications , 2004, Methods of Information in Medicine.

[2]  W. Leow,et al.  DETECTION OF FEMUR AND RADIUS FRACTURES IN X-RAY IMAGES , 2004 .

[3]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[4]  Peter G. B. Enser.  Pictorial information retrieval , 1995 .

[5]  Thomas Martin Deserno,et al.  Automatic medical image annotation in ImageCLEF 2007: Overview, results, and discussion , 2008, Pattern Recognit. Lett..

[6]  Henning Müller,et al.  An Easy Setup for Parallel Medical Image Processing: Using Taverna and ARC , 2009, HealthGrid.

[7]  Wee Kheng Leow,et al.  Hierarchical Classifiers for Detection of Fractures in X-Ray Images , 2007, CAIP.

[8]  Martin Donnelley,et al.  Computer aided long bone fracture detection , 2005, Proceedings of the Eighth International Symposium on Signal Processing and Its Applications, 2005..

[9]  B. G. Weber.  Die Verletzungen des oberen Sprunggelenkes , 1972 .

[10]  Colin C. Venters,et al.  A Review of Content-Based Image Retrieval Systems , 1982 .

[11]  Henning Müller,et al.  Information Fusion for Combining Visual and Textual Image Retrieval , 2010, 2010 20th International Conference on Pattern Recognition.

[12]  Cheng Thao,et al.  GoldMiner: a radiology image search engine. , 2007, AJR. American journal of roentgenology.

[13]  Martin Donnelley,et al.  A CAD System for Long-Bone Segmentation and Fracture Detection , 2008, ICISP.

[14]  James S. Duncan,et al.  Synthesis of Research: Medical Image Databases: A Content-based Retrieval Approach , 1997, J. Am. Medical Informatics Assoc..

[15]  Henning Müller,et al.  Overview of the ImageCLEFmed 2008 Medical Image Retrieval Task , 2008, CLEF.

[16]  Barbara Caputo,et al.  CLEF2007: Image Annotation Task: an SVM-based Cue Integration Approach , 2007, CLEF.

[17]  Hayit Greenspan,et al.  TAU MIPLAB at ImageClef 2008 , 2008, CLEF.

[18]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[19]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[20]  Eugene Kim,et al.  Overview of the ImageCLEFmed 2006 Medical Retrieval and Medical Annotation Tasks , 2006, CLEF.

[21]  Antoine Geissbühler,et al.  3D Case-Based Retrieval for Interstitial Lung Diseases , 2009, MCBR-CDS.

[22]  Ying Chen,et al.  Combining classifiers for bone fracture detection in X-ray images , 2005, IEEE International Conference on Image Processing 2005.

[23]  Philip Bille,et al.  A survey on tree edit distance and related problems , 2005, Theor. Comput. Sci..

[24]  Henry J. Lowe,et al.  Towards knowledge-based retrieval of medical images. The role of semantic indexing, image content representation and knowledge-based retrieval , 1998, AMIA.

[25]  Christian Lovis,et al.  Content-based image retrieval from a database of fracture images , 2007, SPIE Medical Imaging.

[26]  L. Audigé,et al.  Fracture and dislocation classification compendium - 2007: Orthopaedic Trauma Association classification, database and outcomes committee. , 2007, Journal of orthopaedic trauma.

[27]  Eugene Kim,et al.  Overview of the ImageCLEFmed 2006 Medical Retrieval and Annotation Tasks , 2006, CLEF.

[28]  Gustavo Carneiro,et al.  Sparse Flexible Models of Local Features , 2006, ECCV.

[29]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[30]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Hans Burkhardt,et al.  Image classification using cluster cooccurrence matrices of local relational features , 2006, MIR '06.

[32]  O. Ratib,et al.  Casimage Project: A Digital Teaching Files Authoring Environment , 2004, Journal of thoracic imaging.

[33]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[34]  Martin Donnelley,et al.  Automated bone fracture detection , 2005, SPIE Medical Imaging.