Getting Computers to See Information Graphics So Users Do Not Have to

Information graphics such as bar, line and pie charts appear frequently in electronic media and often contain information that is not found elsewhere in documents. Unfortunately, sight-impaired users have difficulty accessing and assimilating information graphics. Our goal is an interactive natural language system that provides effective access to information graphics for sight-impaired individuals. This paper describes how image processing has been applied to transform an information graphic into an XML representation that captures all aspects of the graphic that might be relevant to extracting knowledge from it. It discusses the problems that were encountered in analyzing and categorizing components of the graphic, and the algorithms and heuristics that were successfully applied. The resulting XML representation serves as input to an evidential reasoning component that hypothesizes the message that the graphic was intended to convey.

[1]  Kathleen F. McCoy,et al.  Extending Document Summarization to Information Graphics , 2004 .

[2]  Andrea R. Kennel.  Audiograf: a diagram-reader for the blind , 1996, Assets '96.

[3]  Mark O. Riedl,et al.  A perception/action substrate for cognitive modeling in HCI , 2001, Int. J. Hum. Comput. Stud..

[4]  Ioannis A. Kakadiaris,et al.  Understanding diagrams in technical documents , 1992, Computer.

[5]  Nancy Green,et al.  Incorporating Perceptual Task Effort into the Recognition of Intention in Information Graphics , 2004, Diagrams.

[6]  M. Corio,et al.  Generation of texts for information graphics , 1999 .

[7]  Chew Lim Tan,et al.  Word extraction using irregular pyramid , 2000, IS&T/SPIE Electronic Imaging.

[8]  Jim Hunter,et al.  Recognising Visual Patterns to Communicate Gas Turbine Time-Series Data , 2003 .

[9]  Jonathan Ginzburg,et al.  Proceedings of the Fourth {SIGdial} Workshop on Discourse and Dialogue , 2003 .

[10]  Robert P. Futrelle.  Ambiguity in visual language theory and its role in diagram parsing , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[11]  Nancy Green,et al.  Understanding Information Graphics: A Discourse-Level Problem , 2003, SIGDIAL Workshop.

[12]  Robert P. Futrelle,et al.  Diagram Analysis using Context-Based Constraint Grammars 1 , 1996 .

[13]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[14]  Nancy Green,et al.  Extending Plan Inference Techniques to Recognize Intentions in Information Graphics , 2003, User Modeling.

[15]  Robert P. Futrelle,et al.  Summarization of Diagrams in Documents , 1999 .