Adaptive Layout for Interactive Documents

In many application domains there is a strong need to produce content both for traditional print media and for interactive media. In order to fully benefit from digital devices, online documents must provide mechanisms to support interactivity and for the personalization of content. Thus, powerful authoring tools as well as flexible layout techniques are needed to display dynamic information effectively. In this paper we present a novel approach to template-based design of complex non-grid layouts and propose a unique combination of constraint-based and force-based layout mechanisms. Our authoring tool enables designer to specify designs pattern with overlapping content elements that are transformed into layout constraints and force systems by the layout engine. The smooth integration of interactive illustrations into complex layouts also induces constraints to achieve temporal coherent layout. We propose algorithms to achieve graceful layouts even when the space consumption of content elements significantly differs from the defaults specified in layout templates.

[1]  James D. Hollan,et al.  Zooming Web browser , 1996, Electronic Imaging.

[2]  Timo Götzelmann,et al.  Practical Illustration of Text: Customized Search, View Selection, and Annotation , 2006, Mensch & Computer.

[3]  Rolf Ernst,et al.  An approach to automatic display layout using combinatorial optimization algorithms , 1995, Softw. Pract. Exp..

[4]  Elizabeth D. Liddy,et al.  Advances in Automatic Text Summarization , 2001, Information Retrieval.

[5]  Manojit Sarkar,et al.  Graphical fisheye views of graphs , 1992, CHI.

[6]  Bill N. Schilit,et al.  Digestor: Device-Independent Access to the World Wide Web , 1997, Comput. Networks.

[7]  John R. Smith,et al.  Adapting Multimedia Internet Content for Universal Access , 1999, IEEE Trans. Multim..

[8]  Steven K. Feiner,et al.  Dynamic space management for user interfaces , 2000, UIST '00.

[9]  Toshiyuki Masui,et al.  Evolutionary learning of graph layout constraints from examples , 1994, UIST '94.

[10]  Richard John Beach Setting tables and illustrations with style , 1985 .

[11]  Thomas Ertl,et al.  Transparency in Interactive Technical Illustrations , 2002, Comput. Graph. Forum.

[12]  Gail E. Kaiser,et al.  Automating Content Extraction of HTML Documents , 2005, World Wide Web.

[13]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[14]  Cécile Roisin,et al.  Constraints for Multimedia Documents , 2000, PACPL.

[15]  Steven K. Feiner,et al.  A grid-based approach to automating display layout , 1998 .

[16]  James D. Hollan,et al.  Pad++: a zoomable graphical interface system , 1995, CHI 95 Conference Companion.

[17]  Atsushi Takahashi,et al.  Air-pressure-model-based fast algorithms for general floorplan , 1998, Proceedings of 1998 Asia and South Pacific Design Automation Conference.

[18]  Frank Wagner,et al.  A packing problem with applications to lettering of maps , 1991, SCG '91.

[19]  P. Roetling,et al.  Aesthetic measures for automated document layout , 2004, DocEng '04.

[20]  Joe Marks,et al.  Automatic Yellow-Pages pagination and layout , 1997, J. Heuristics.

[21]  W. Hollinshead,et al.  A TEXTBOOK OF ANATOMY , 1925 .

[22]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[23]  Matti Pietikaeinen,et al.  Automated document content characterization for a multimedia document retrieval system , 1997, Other Conferences.

[24]  Joe Marks,et al.  An empirical study of algorithms for point-feature label placement , 1995, TOGS.

[25]  Timo Götzelmann,et al.  Adaptive Labeling for Illustrations , 2005 .

[26]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[27]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[28]  Michelangelo Ceci,et al.  Correcting the document layout: a machine learning approach , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[29]  G. Nunberg The non-uniqueness of semantic solutions: Polysemy , 1979 .

[30]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[31]  M. Sheelagh T. Carpendale,et al.  Integrating expanding annotations with a 3D explosion probe , 2004, AVI.

[32]  Gabriella Sanniti di Baja,et al.  Finding local maxima in a pseudo-Euclidian distance transform , 1988, Comput. Vis. Graph. Image Process..

[33]  Jock D. Mackinlay,et al.  The document lens , 1993, UIST '93.

[34]  Alfred Bork,et al.  Multimedia in Learning , 2001 .

[35]  Elena Deza,et al.  Dictionary of distances , 2006 .

[36]  Steven K. Feiner,et al.  Windows on the world: 2D windows for 3D augmented reality , 1993, UIST '93.

[37]  Kozo Sugiyama,et al.  A Simple and Unified Method for Drawing Graphs: Magnetic-Spring Algorithm , 1994, GD.

[38]  Alan K. Mackworth The Logic of Constraint Satisfaction , 1991, Artif. Intell..

[39]  Brad A. Myers,et al.  Marquise: creating complete user interfaces by demonstration , 1993, CHI '93.

[40]  Per-Erik Danielson A new shape factor , 1978 .

[41]  Jock D. Mackinlay,et al.  The perspective wall: detail and context smoothly integrated , 1991, CHI.

[42]  Nikolaus F. Troje,et al.  Paper windows: interaction techniques for digital paper , 2005, CHI.

[43]  L. Weitzman,et al.  Grammar-based articulation for multimedia document design : Special section on ACM Multimedia 1994 , 1996 .

[44]  David Harel,et al.  Drawing graphs with non-uniform vertices , 2002, AVI '02.

[45]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[46]  Christine Reid,et al.  The Myth of the Paperless Office , 2003, J. Documentation.

[47]  Bertin Klein,et al.  Human-centered interaction with documents , 2006, HCM '06.

[48]  J. Laidlaw,et al.  ANATOMY OF THE HUMAN BODY , 1967, The Ulster Medical Journal.

[49]  Timo Götzelmann,et al.  Agent-Based Annotation of Interactive 3D Visualizations , 2006, Smart Graphics.

[50]  Satoshi Matsuoka,et al.  Declarative programming of graphical interfaces by visual examples , 1992, UIST '92.

[51]  Raimondo Schettini,et al.  Self-Adaptive Image Cropping for Small Displays , 2007, IEEE Transactions on Consumer Electronics.

[52]  Berna Erol,et al.  Multimedia thumbnails for documents: implementation and demonstration , 2006, MM '06.

[53]  G. Williams WHAT IS A DOCUMENT , 1948 .

[54]  H. Schumann,et al.  Maintenance support: case study for a multimodal mobile user interface , 2006, Electronic Imaging.

[55]  Wei-Ying Ma,et al.  Extracting Content Structure for Web Pages Based on Visual Representation , 2003, APWeb.

[56]  Thomas Strothotte,et al.  Illustrative Browsing: A New Method of Browsing in Long On-line Texts , 1999, INTERACT.

[57]  A. Paivio,et al.  Dual coding theory and education , 1991 .

[58]  Knut Hartmann,et al.  Label Layout for Interactive 3D Illustrations , 2005, J. WSCG.

[59]  Ioannis G. Tollis,et al.  A unified approach to labeling graphical features , 1998, SCG '98.

[60]  Aarne Ranta,et al.  XML and Multilingual Document Authoring: Convergent Trends , 2000, COLING.

[61]  David Salesin,et al.  Adaptive Document Layout via Manifold Content , 2003 .

[62]  Robert Z. Calvert Improvisational Design: Continuous, Responsive Digital Communication , 2003 .

[63]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[64]  Chris Stolte,et al.  Rendering effective route maps: improving usability through generalization , 2001, SIGGRAPH.

[65]  Annie Piolat,et al.  Effects of screen presentation on text reading and revising , 1997, Int. J. Hum. Comput. Stud..

[66]  David Salesin,et al.  Adaptive grid-based document layout , 2003, ACM Trans. Graph..

[67]  Jock D. Mackinlay,et al.  Applying a theory of graphical presentation to the graphic design of user interfaces , 1988, UIST '88.

[68]  Michel Schmitt,et al.  Some examples of algorithm analysis in computational geometry by means of mathematical morphological techniques , 1988, Geometry and Robotics.

[69]  Timo Götzelmann,et al.  Annotating Images through Adaptation: An Integrated Text Authoring and Illustration Framework , 2007, J. WSCG.

[70]  Knut Hartmann,et al.  Illustrative shadows: integrating 3D and 2D information displays , 2003, IUI '03.

[71]  Heidrun Schumann,et al.  Particle-based labeling: Fast point-feature labeling without obscuring other visual features , 2008, IEEE Transactions on Visualization and Computer Graphics.

[72]  E. Bier,et al.  Zoomable user interface for in-depth reading , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[73]  Roberto Cipolla,et al.  Skeletonization using an extended Euclidean distance transform , 1995, Image Vis. Comput..

[74]  Andreas Paepcke,et al.  Accordion summarization for end-game browsing on PDAs and cellular phones , 2001, CHI.

[75]  Louis Weitzman,et al.  Relational grammars for interactive design , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[76]  A. ROSENFELD,et al.  Distance functions on digital pictures , 1968, Pattern Recognit..

[77]  Thomas Ertl,et al.  Interactive Cutaway Illustrations , 2003, Comput. Graph. Forum.

[78]  Heidrun Schumann,et al.  Adaptive Labeling for Interactive Mobile Information Systems , 2006, Tenth International Conference on Information Visualisation (IV'06).

[79]  Polina Golland,et al.  AnatomyBrowser: A Framework for Integration of Medical Information , 1998, MICCAI.

[80]  Alex White,et al.  The Elements of Graphic Design: Space, Unity, Page Architecture, and Type , 2002 .

[81]  Fabio Paternò,et al.  CTTE: Support for Developing and Analyzing Task Models for Interactive System Design , 2002, IEEE Trans. Software Eng..

[82]  Knut Hartmann,et al.  Floating Labels: Applying Dynamic Potential Fields for Label Layout , 2004, Smart Graphics.

[83]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[84]  Peter J. Stuckey,et al.  The Cassowary linear arithmetic constraint solving algorithm , 2001, TCHI.

[85]  Billie F. Spencer,et al.  Models for hysteresis and application to structural control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[86]  Donato Malerba,et al.  Transforming paper documents into XML format with WISDOM++ , 2001, International Journal on Document Analysis and Recognition.

[87]  Erik Wästlund,et al.  The effect of page layout on mental workload: A dual-task experiment , 2008, Comput. Hum. Behav..

[88]  Fabio Paternò,et al.  ConcurTaskTrees: A Diagrammatic Notation for Specifying Task Models , 1997, INTERACT.

[89]  Thomas Kämpke,et al.  Label placement for dynamic objects , 2003 .

[90]  Wallace Chigona,et al.  Shadows with a Message , 2003, Smart Graphics.

[91]  Bernhard Preim,et al.  Coherent Zooming of Illustrations with 3D-Graphics and Text , 1997, Graphics Interface.

[92]  Abigail Sellen,et al.  Affordances for manipulation of physical versus digital media on interactive surfaces , 2007, CHI.

[93]  Steven K. Feiner,et al.  Information at a Glance , 2002, IEEE Computer Graphics and Applications.

[94]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[95]  Michael Frederick Plass,et al.  Optimal pagination techniques for automatic typesetting systems , 1981 .

[96]  Alexander Wolff,et al.  Boundary labeling: Models and efficient algorithms for rectangular maps , 2004, Comput. Geom..

[97]  Michael K. Buckland,et al.  What is a digital document , 1998 .

[98]  Caroline Brun,et al.  Document structure and multilingual authoring , 2000, INLG.

[99]  Edward M. Reingold,et al.  Graph drawing by force‐directed placement , 1991, Softw. Pract. Exp..

[100]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[101]  Christopher B. Jones,et al.  A Prolog Rule‐Based System for cartographic Name Placement , 1990, Comput. Graph. Forum.

[102]  Clayton Lewis,et al.  A problem-oriented classification of visualization techniques , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[103]  Ben Shneiderman,et al.  LifeLines: using visualization to enhance navigation and analysis of patient records , 1998, AMIA.

[104]  Alan Borning,et al.  Constraint-based document layout for the Web , 2000, Multimedia Systems.

[105]  Alan Borning,et al.  A constraint extension to scalable vector graphics , 2001, WWW '01.

[106]  Gary Marchionini,et al.  Visualizing websites using a hierarchical table of contents browser:WebTOC , 1998 .

[107]  Timo Götzelmann,et al.  Metrics for Functional and Aesthetic Label Layouts , 2005, Smart Graphics.

[108]  Louis Weitzman,et al.  Grammar-based articulation for multimedia document design , 1998, Multimedia Systems.

[109]  Joe Marks,et al.  The Computational Complexity of Cartographic Label Placement , 1991 .

[110]  Wei-Ying Ma,et al.  VIPS: a Vision-based Page Segmentation Algorithm , 2003 .

[111]  Pat Hanrahan,et al.  Designing effective step-by-step assembly instructions , 2003, ACM Trans. Graph..

[112]  Louis Weitzman,et al.  Automatic presentation of multimedia documents using relational grammars , 1994, MULTIMEDIA '94.

[113]  Rolf Klein,et al.  On the Pagination of Complex Documents , 2003, Computer Science in Perspective.

[114]  Lutz Plümer,et al.  FAST SCREEN MAP LABELING – DATA-STRUCTURES AND ALGORITHMS , 2003 .

[115]  B. Rudolph,et al.  Preparing Scientific Illustrations: A Guide to Better Posters, Presentations, and Publications: Mary Helen Briscoe 1996 Springer Inc. 536 Broadway, New York, NY 10012- 3915, USA Softcover: $29.95 , 1997 .

[116]  Bjørn N. Freeman-Benson,et al.  Multi‐way versus one‐way constraints in user interfaces: Experience with the deltablue algorithm , 1993, Softw. Pract. Exp..

[117]  M. Clark,et al.  The Oxford-Duden pictorial English dictionary , 1995 .

[118]  Mary C. Dyson,et al.  The legibility of screen formats: are three columns better than one? , 1997, Comput. Graph..

[119]  Ivan Poupyrev,et al.  An Introduction to 3-D User Interface Design , 2001, Presence: Teleoperators & Virtual Environments.

[120]  John C. Nash,et al.  The (Dantzig) simplex method for linear programming , 2000, Comput. Sci. Eng..

[121]  Jean-Daniel Fekete,et al.  Excentric labeling: dynamic neighborhood labeling for data visualization , 1999, CHI '99.

[122]  Donald G. Bailey,et al.  An Efficient Euclidean Distance Transform , 2004, IWCIA.

[123]  M. Schwarz,et al.  Otto-von-Guericke-University of Magdeburg , 2007 .

[124]  Jette Hyldegaard,et al.  My e-journal - exploring the usefulness of personalized access to scholarly articles and services , 2004, Inf. Res..

[125]  Xiaobo Wang,et al.  Generating Customized Layouts , 1995, Graph Drawing.

[126]  Frank Y. Shih,et al.  Skeletonization And Distance Transformation By Greyscale Morphology , 1988, Other Conferences.

[127]  Ronald Azuma,et al.  Evaluating label placement for augmented reality view management , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[128]  Wilfred J. Hansen,et al.  Reading and writing with computers: a framework for explaining differences in performance , 1988, CACM.

[129]  Donald E. Knuth,et al.  Breaking paragraphs into lines , 1981, Softw. Pract. Exp..

[130]  David Salesin,et al.  Interactive Image-Based Exploded View Diagrams , 2004, Graphics Interface.

[131]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[132]  Bernhard Preim,et al.  Virtual 3D Jigsaw Puzzles: Studying the Effect of Exploring Spatial Relations with Implicit Guidance , 2002, MuC.

[133]  Bernhard Preim,et al.  An Evaluation of Interaction Techniques for the Exploration of 3D-Illustrations , 1999, Software-Ergonomie.

[134]  E. Imhof Positioning Names on Maps , 1975 .

[135]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[136]  Takahiro Hara,et al.  Image classification for mobile web browsing , 2006, WWW '06.

[137]  Michael K. Buckland,et al.  Information as thing , 1991, J. Am. Soc. Inf. Sci..

[138]  Winfried Graf,et al.  Constraint-Based Graphical Layout of Multimodal Presentations , 1998, Advanced Visual Interfaces.

[139]  H. Reeves,et al.  The Guild handbook of scientific illustration , 1991 .

[140]  Chang-Sung Jeong,et al.  A straight line detection using principal component analysis , 2006, Pattern Recognit. Lett..

[141]  Timo Götzelmann,et al.  Form Follows Function: Aesthetic Interactive Labels , 2005, CAe.