Improving the Accessibility of Line Graphs in Multimodal Documents

This paper describes our work on improving access to the content of multimodal documents containing line graphs in popular media for people with visual impairments. We provide an overview of our implemented system, including our method for recognizing and conveying the intended message of a line graph. The textual description of the graphic generated by our system is presented at the most relevant point in the document. We also describe ongoing work into obtaining additional propositions that elaborate on the intended message, and examine the potential benefits of analyzing the text and graphical content together in order to extend our system to produce summaries of entire multimodal documents.

[1]  Kathleen F. McCoy,et al.  Generating Textual Summaries of Bar Charts , 2008, INLG.

[2]  Harold E. Wefold.  Computer graphics for the blind , 1976, SIGC.

[3]  Stephen Brewster,et al.  Web-based Multimodal Graphs for Visually Impaired People , 2002 .

[4]  Dominic Widdows,et al.  Orthogonal Negation in Vector Spaces for Modelling Word-Meanings and Document Retrieval , 2003, ACL.

[5]  David D. McDonald.  An Efficient Chart-based Algorithm for Partial-Parsing of Unrestricted Texts , 1992, ANLP.

[6]  W. Bruce Croft,et al.  Passage retrieval based on language models , 2002, CIKM '02.

[7]  Michael Elhadad,et al.  An Overview of SURGE: a Reusable Comprehensive Syntactic Realization Component , 1996, INLG.

[8]  David McDonald,et al.  'If you've heard it, you can say it' - Towards an Account of Expressibility , 2010, INLG.

[9]  K.E. Barner,et al.  Visual to Tactile Conversion of Vector Graphics , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[10]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[11]  Udo Hahn,et al.  Text condensation as knowledge base abstraction , 1988, [1988] Proceedings. The Fourth Conference on Artificial Intelligence Applications.

[12]  Daniel Marcu,et al.  The rhetorical parsing, summarization, and generation of natural language texts , 1998 .

[13]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[14]  Enrico Pontelli,et al.  Multimodal Presentation of Two-Dimensional Charts: An Investigation Using Open Office XML and Microsoft Excel , 2010, TACC.

[15]  Bruce N. Walker,et al.  Digitizer auditory graph: making graphs accessible to the visually impaired , 2010, CHI Extended Abstracts.

[16]  David D. McDonald.  Issues in the representation of real texts: the design of KRISP , 2000 .

[17]  Stephanie Elzer Schwartz,et al.  Information graphics: an untapped resource for digital libraries , 2006, SIGIR.

[18]  Gitte Lindgaard,et al.  Improving accessibility to statistical graphs: the iGraph-Lite system , 2007, Assets '07.

[19]  ChengXiang Zhai,et al.  Statistical Language Models for Information Retrieval , 2008, NAACL.

[20]  Francine Chen,et al.  A trainable document summarizer , 1995, SIGIR '95.

[21]  David D. McDonald.  Partially Saturated Referents as a Source of Complexity in Semantic Interpretation , 2000 .

[22]  Vibhu O. Mittal,et al.  Ultra-summarization (poster abstract): a statistical approach to generating highly condensed non-extractive summaries , 1999, SIGIR '99.

[23]  Shibamouli Lahiri,et al.  Generating synopses for document-element search , 2009, CIKM.

[24]  Kathleen F. McCoy,et al.  A Discourse-Aware Graph-Based Content-Selection Framework , 2010, INLG.

[25]  Vibhu O. Mittal,et al.  Ultra-Summarization: A Statistical Approach to Generating Highly Condensed Non-Extractive Summaries (poster abstract). , 1998, SIGIR 1999.

[26]  Ingrid Zukerman,et al.  The automated understanding of simple bar charts , 2011, Artif. Intell..

[27]  Kathleen F. McCoy,et al.  A Corpus of Human-written Summaries of Line Graphs , 2011 .

[28]  James L. Alty,et al.  Communicating graphical information to blind users using music: the role of context , 1998, CHI.

[29]  Evgueni A. Haroutunian,et al.  Information Theory and Statistics , 2011, International Encyclopedia of Statistical Science.