Describing Abstraction in Rendered Images through Figure Captions

We analyze illustration and abstraction techniques used in rendered images. We argue that it is important to convey these techniques to viewers of such images to enhance the process of image understanding. This leads us to derive methods for automatically generating figure captions for rendered images which describe the abstraction carried out. We apply this concept to computer generated anatomic illustrations. Strategies for the content selection for figure captions, for setting user preferences and for updating figure captions and for interaction with figure captions are described. The paper also describes a prototypical implementation.

[1]  Wolfgang Wahlster,et al.  Plan-Based Integration of Natural Language and Graphics Generation , 1993, Artif. Intell..

[2]  Thomas Strothotte,et al.  Computational visualization - graphics, abstraction, and interactivity , 2011 .

[3]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[4]  Ehud Reiter,et al.  NLG vs. Templates , 1995, ArXiv.

[5]  Johanna D. Moore,et al.  Generating Explanatory Captions for Information Graphics , 1995, IJCAI.

[6]  Steffen Staab,et al.  "Tall", "Good", "High" - Compared to What? , 1997, IJCAI.

[7]  Chris Mellish,et al.  Optimizing the Costs and Benefits of Natural Language Generation , 1993, International Joint Conference on Artificial Intelligence.

[8]  Johannes Sobotta,et al.  Sobotta Atlas der Anatomie des Menschen , 1988 .

[9]  Stefan Schlechtweg-Dorendorf,et al.  Interaction and Focus: Towards a Coherent Degree of Detail in Graphics, Captions and Text , 1999, SimVis.

[10]  Bernhard Preim,et al.  Figure captions in visual interfaces , 1998, AVI '98.

[11]  David R. Forsey,et al.  CONSISTENCY OF RENDERED IMAGES AND THEIR TEXTUAL LABELS , 1995 .

[12]  Bernhard Preim,et al.  Illustrating Anatomic Models - A Semi-Interactive Approach , 1996, VBC.

[13]  Johanna D. Moore,et al.  Describing Complex Charts in Natural Language: A Caption Generation System , 1998, CL.

[14]  Emanuel G. Noik,et al.  A Space of Presentation Emphasis Techniques for Visualizing Graphs , 1994 .

[15]  Knut Hartmann,et al.  Dynamic Visual Emphasis in Interactive Technical Documentation , 1998 .

[16]  Steven K. Feiner,et al.  Automated generation of intent-based 3D Illustrations , 1991, SIGGRAPH.

[17]  John Levine,et al.  Automatic generation of technical documentation , 1994, Appl. Artif. Intell..

[18]  David Salesin,et al.  Scale-dependent reproduction of pen-and-ink illustrations , 1996, SIGGRAPH.

[19]  Kaufman,et al.  A New Color-Namiing System for Graphics Languages , 1982, IEEE Computer Graphics and Applications.

[20]  Stephan Busemann,et al.  Best-First Surface Realization , 1996, INLG.

[21]  Thomas Rist,et al.  AWI: a workbench for semi-automated illustration design , 1994, AVI '94.

[22]  Helmut Horacek.  A New Algorithm For Generating Referential Descriptions , 1996, ECAI.

[23]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues: Capturing Intentional and Rhetorical Information , 1993, CL.

[24]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[25]  Debra T. Burhans,et al.  Visual Semantics: Extracting Visual information from Text Accompanying Pictures , 1994, AAAI.

[26]  William C. Mann,et al.  RHETORICAL STRUCTURE THEORY: A THEORY OF TEXT ORGANIZATION , 1987 .

[27]  Robert M. Bernard.  Using extended captions to improve learning from instructional illustrations , 1990, Br. J. Educ. Technol..

[28]  Ernst Gombrich,et al.  The image and the eye : further studies in the psychology of pictorial representation , 1985 .

[29]  Steven K. Feiner,et al.  Apex: An Experiment in the Automated Creation of Pictorial Explanations , 1985, IEEE Computer Graphics and Applications.

[30]  David R. Forsey,et al.  How to Render Frames and Influence People , 1994, Comput. Graph. Forum.

[31]  Lyn Bartram,et al.  A continuously variable zoom for navigating large hierarchical networks , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[32]  Bernhard Preim,et al.  Interaktive Illustrationen und Animationen zur Erklärung komplexer räumlicher Zusammenhänge , 1998 .

[33]  Thomas Rist,et al.  Wissensbasierte Verfahren für den automatischen Entwurf von Gebrauchsgraphik in der technischen Dokumentation , 1996, DISKI.

[34]  Kathleen McKeown,et al.  Text generation: using discourse strategies and focus constraints to generate natural language text , 1985 .

[35]  Ernst Gombrich,et al.  The Image and the Eye , 1982 .

[36]  Steven K. Feiner,et al.  Automating the generation of coordinated multimedia explanations , 1991, Computer.

[37]  M. Sheelagh T. Carpendale,et al.  3-dimensional pliable surfaces: for the effective presentation of visual information , 1995, UIST '95.

[38]  Dietmar F. Rösner,et al.  Visdok: Ein Ansatz zur interaktiven Nutzung von technischer Dokumentation , 1998, SimVis.

[39]  Michael White,et al.  EXEMPLARS: A Practical, Extensible Framework For Dynamic Text Generation , 1998, INLG.

[40]  Bernhard Preim,et al.  Coherent Zooming of Illustrations with 3D-Graphics and Text , 1997, Graphics Interface.

[41]  Dr.-Ing. Christine Strothotte,et al.  Seeing Between the Pixels , 1997, Springer Berlin Heidelberg.