Understanding Information Graphics: A Discourse-Level Problem

Keywords: graphics, understanding, discourse , plan-based models Information graphics that appear in newspapers and magazines generally have a message that the viewer is intended to recognize. This paper argues that understanding such information graphics is a discourse-level problem. In particular, it requires assimilating information from multiple knowledge sources to recognize the intended message of the graphic, just as recognizing intention in text does. Moreover, when an article is composed of text and graphics, the intended message of the information graphic (its discourse intention) must be integrated into the discourse structure of the surrounding text and contributes to the overall discourse intention of the article. This paper describes how we extend plan-based techniques that have been used for understanding traditional discourse to the understanding of information graphics. This work is part of a project to develop an interactive natural language system that provides sight-impaired users with access to information graphics.

[1]  H. Grice.  Utterer's meaning and intentions , 1969 .

[2]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[3]  John R. Searle,et al.  Speech Acts: An Essay in the Philosophy of Language , 1970 .

[4]  C. Raymond Perrault,et al.  A Plan-Based Analysis of Indirect Speech Act , 1980, CL.

[5]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[6]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[7]  James F. Allen,et al.  A Plan Recognition Model for Subdialogues in Conversations , 1987, Cogn. Sci..

[8]  James F. Allen,et al.  A Plan Recognition Model for Subdialogues in Conversations , 1987, Cogn. Sci..

[9]  S. Kosslyn.  Understanding charts and graphs , 1989 .

[10]  Sandra Carberry,et al.  Plan Recognition in Natural Language Dialogue , 1990 .

[11]  Bonnie E. John,et al.  Toward an Engineering Model of Stimulus-Response Compatibility , 1990 .

[12]  Peter B. L. Meijer,et al.  An experimental system for auditory image representations , 1992, IEEE Transactions on Biomedical Engineering.

[13]  Gerald L. Lohse,et al.  A Cognitive Model for Understanding Graphical Perception , 1993, Hum. Comput. Interact..

[14]  Robert P. Goldman,et al.  A Bayesian Model of Plan Recognition , 1993, Artif. Intell..

[15]  Christopher D. Wickens,et al.  The Proximity Compatibility Principle: Its Psychological Foundation and Relevance to Display Design , 1995, Hum. Factors.

[16]  Andrea R. Kennel.  Audiograf: a diagram-reader for the blind , 1996, Assets '96.

[17]  Bonnie L. Webber,et al.  A Bias towards Relevance: Recognizing Plans where Goal Minimization Fails , 1996, AAAI/IAAI, Vol. 2.

[18]  Vibhu O. Mittal.  Visual Prompts and Graphical Design: A Framework for Exploring the Design Space of 2-D Charts and Graphs , 1997, AAAI/IAAI.

[19]  M. Corio,et al.  Generation of texts for information graphics , 1999 .

[20]  Lynn Lambert,et al.  A Process Model for Recognizing Communicative Acts and Modeling Negotiation Subdialogues , 1999, Comput. Linguistics.

[21]  Kathleen F. McCoy,et al.  Efficient text summarization using lexical chains , 2000, IUI '00.

[22]  Steven F. Roth,et al.  Mapping communicative goals into conceptual tasks to generate graphics in discourse , 2000, IUI '00.

[23]  Nancy Green,et al.  Extending Plan Inference Techniques to Recognize Intentions in Information Graphics , 2003, User Modeling.

[24]  Jim Hunter,et al.  Recognising Visual Patterns to Communicate Gas Turbine Time-Series Data , 2003 .

[25]  Sandra Carberry,et al.  Exploiting Cognitive Psychology Research for Recognizing Intention in Information Graphics , 2003 .

[26]  Liliana Ardissono,et al.  Using dynamic user models in the recognition of the plans of the user , 2005, User Modeling and User-Adapted Interaction.