Stitched Multipanel Biomedical Figure Separation

We present a novel technique to separate subpanels from stitched multipanel figures appearing in biomedical research articles. Since such figures may comprise images from different imaging modalities, separating them is a critical first step for effective biomedical content-based image retrieval (CBIR). The method applies local line segment detection based on the gray-level pixel changes. It then applies a line vectorization process that connects prominent broken lines along the subpanel boundaries while eliminating insignificant line segments within the subpanels. We have validated our fully automatic technique on a subset of stitched multipanel biomedical figures extracted from articles within the Open Access subset of PubMed Central repository, and have achieved precision and recall of 81.22% and 85.08%, respectively.

[1]  Dragutin Petkovic,et al.  Content-based representation and retrieval of visual media: A state-of-the-art review , 1996, Multimedia Tools and Applications.

[2]  Cris L. Luengo Hendriks,et al.  Discrete Morphology with Line Structuring Elements , 2003, CAIP.

[3]  C. Wagner-Mann,et al.  If a picture is worth a thousand words, what is a trauma computerized tomography panel worth? , 2007, American journal of surgery.

[4]  Henning Müller.  Medical (Visual) Information Retrieval , 2012, PROMISE Winter School.

[5]  R. Joe Stanley,et al.  Automatic segmentation of subfigure image panels for multimodal biomedical document retrieval , 2011, Electronic Imaging.

[6]  Manabu Torii,et al.  A framework for biomedical figure segmentation towards image-based document retrieval , 2013, BMC Systems Biology.

[7]  Henning Müller,et al.  Separating compound figures in journal articles to allow for subfigure classification , 2013, Medical Imaging.

[8]  Daekeun You,et al.  Interactive cross and multimodal biomedical image retrieval based on automatic region-of-interest (ROI) identification and classification , 2014, International Journal of Multimedia Information Retrieval.

[9]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[10]  George R. Thoma,et al.  Multimodal biomedical image indexing and retrieval using descriptive text and global feature mapping , 2013, Information Retrieval.

[11]  Hayit Greenspan,et al.  Content-Based Image Retrieval in Radiology: Current Status and Future Directions , 2010, Journal of Digital Imaging.

[12]  Henning Müller,et al.  Overview of the ImageCLEF 2013 Medical Tasks , 2013, CLEF.

[13]  George R. Thoma,et al.  Design and Development of a Multimodal Biomedical Information Retrieval System , 2012, J. Comput. Sci. Eng..

[14]  Jie Yao,et al.  Searching online journals for fluorescence microscope images depicting protein subcellular location patterns , 2001, Proceedings 2nd Annual IEEE International Symposium on Bioinformatics and Bioengineering (BIBE 2001).

[15]  Hong Yu,et al.  Towards Answering Biological Questions with Experimental Evidence: Automatically Identifying Text that Summarize Image Content in Full-Text Articles , 2006, AMIA.

[16]  Carey Phillips,et al.  The Zebrafish DVD Exchange Project: a bioinformatics initiative. , 2004, Methods in cell biology.

[17]  Antoine Geissbühler,et al.  A Review of Content{Based Image Retrieval Systems in Medical Applications { Clinical Bene(cid:12)ts and Future Directions , 2022 .

[18]  J. C. R. Licklider.  A picture is worth a thousand words: and it costs... , 1969, AFIPS '69 (Spring).

[19]  Rafael Grompone von Gioi,et al.  LSD: a Line Segment Detector , 2012, Image Process. Line.

[20]  Henning Müller,et al.  Overview of the ImageCLEF 2012 Medical Image Retrieval and Classification Tasks , 2012, CLEF.

[21]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[22]  Henning Müller,et al.  Overview of the CLEF 2011 Medical Image Classification and Retrieval Tasks , 2011, CLEF.

[23]  Daekeun You,et al.  Image retrieval from scientific publications: Text and image content processing to separate multipanel figures , 2013, J. Assoc. Inf. Sci. Technol..