A system for understanding imaged infographics and its applications

Information graphics, or infographics, are visual representations of information, data or knowledge. Understanding of infographics in documents is a relatively new research problem, which becomes more challenging when infographics appear as raster images. This paper describes technical details and practical applications of the system we built for recognizing and understanding imaged infographics located in document pages. To recognize infographics in raster form, both graphical symbol extraction and text recognition need to be performed. The two kinds of information are then auto-associated to capture and store the semantic information carried by the infographics. Two practical applications of the system are introduced in this paper, including supplement to traditional optical character recognition (OCR) system and providing enriched information for question answering (QA). To test the performance of our system, we conducted experiments using a collection of downloaded and scanned infographic images. Another set of scanned document pages from the University of Washington document image database were used to demonstrate how the system output can be used by other applications. The results obtained confirm the practical value of the system.

[1]  Jinxi Xu,et al.  Evaluation of an extraction-based approach to answering definitional questions , 2004, SIGIR '04.

[2]  Kathleen F. McCoy,et al.  Extending Document Summarization to Information Graphics , 2004 .

[3]  Toyohide Watanabe,et al.  Layout-Based Approach for Extracting Constructive Elements of Bar-Charts , 1997, GREC.

[4]  Chew Lim Tan,et al.  A multi-level component grouping algorithm and its applications , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[5]  Chew Lim Tan,et al.  Chart Image Classification Using Multiple-Instance Learning , 2007, 2007 IEEE Workshop on Applications of Computer Vision (WACV '07).

[6]  Eduard H. Hovy,et al.  Learning surface text patterns for a Question Answering System , 2002, ACL.

[7]  Peter L. Brooks,et al.  Visualizing data , 1997 .

[8]  Ching Y. Suen,et al.  Logical Block Labeling for Diverse Types of Document Images , 1999 .

[9]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[10]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[11]  Chew Lim Tan,et al.  Learning-based scientific chart recognition , 2001 .

[12]  Pan Shi-yan.  A Form Frame-Line Detection Algorithm Based on Directional Single-Connected Chain , 2002 .

[13]  Tat-Seng Chua,et al.  Unsupervised learning of soft patterns for generating definitions from online news , 2004, WWW '04.

[14]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[15]  Fei Wang,et al.  NPIC: Hierarchical Synthetic Image Classification Using Image Search and Generic Features , 2006, CIVR.

[16]  Chew Lim Tan,et al.  Model-Based Chart Image Recognition , 2003, GREC.

[17]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Chew Lim Tan,et al.  Hough-based model for recognizing bar charts in document images , 2000, IS&T/SPIE Electronic Imaging.

[19]  T. Breuel.  Layout Analysis based on Text Line Segment Hypotheses , 2003 .

[20]  Bart Lamiroy,et al.  Text/Graphics Separation Revisited , 2002, Document Analysis Systems.

[21]  Jim Hunter,et al.  Recognising Visual Patterns to Communicate Gas Turbine Time-Series Data , 2003 .

[22]  Nancy Green,et al.  Understanding Information Graphics: A Discourse-Level Problem , 2003, SIGDIAL Workshop.

[23]  W. Cleveland,et al.  The elements of graphing data , 1985 .

[24]  Ioannis A. Kakadiaris,et al.  Understanding diagrams in technical documents , 1992, Computer.

[25]  Rohit J. Kate,et al.  Using String-Kernels for Learning Semantic Parsers , 2006, ACL.

[26]  Z. Yanping,et al.  Coordinate systems reconstruction for graphical documents by Hough-feature clustering and geometric analysis , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..