Fluid interaction with high-resolution wall-size displays

This paper describes new interaction techniques for direct pen-based interaction on the Interactive Mural, a large (6′x3.5′) high resolution (64 dpi) display. They have been tested in a digital brainstorming tool that has been used by groups of professional product designers. Our "interactive wall" metaphor for interaction has been guided by several goals: to support both free-hand sketching and high-resolution materials, such as images, 3D models and GUI application windows; to present a visual appearance that does not clutter the content with control devices; and to support fluid interaction, which minimizes the amount of attention demanded and interruption due to the mechanics of the interface. We have adapted and extended techniques that were developed for electronic whiteboards and generalized the use of the FlowMenu to execute a wide variety of actions in a single pen stroke, While these techniques were designed for a brainstorming tool, they are very general and can be used in a wide variety of application domains using interactive surfaces.

[1]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[2]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[3]  Adam Finkelstein,et al.  Building and Using A Scalable Display Wall System , 2000, IEEE Computer Graphics and Applications.

[4]  Daniel G. Bobrow,et al.  Beyond the chalkboard: computer support for collaboration and problem solving in meetings , 1988, CACM.

[5]  Gordon Stoll,et al.  WireGL: a scalable graphics system for clusters , 2001, SIGGRAPH.

[6]  Thomas P. Moran,et al.  Pen-based interaction techniques for organizing material on an electronic whiteboard , 1997, UIST '97.

[7]  Homan Igehy,et al.  The design of a parallel graphics interface , 1998, SIGGRAPH.

[8]  Darren Leigh,et al.  DiamondTouch: a multi-user touch technology , 2001, UIST '01.

[9]  Jay F. Nunamaker,et al.  Electronic meeting systems , 1991, CACM.

[10]  William Buxton,et al.  The design of a GUI paradigm based on tablets, two-hands, and transparency , 1997, CHI.

[11]  Thomas P. Moran,et al.  Tailorable domain objects as meeting tools for an electronic whiteboard , 1998, CSCW '98.

[12]  A. Fox,et al.  Integrating information appliances into an interactive workspace , 2000, IEEE Computer Graphics and Applications.

[13]  Takeo Igarashi,et al.  Speed-dependent automatic zooming for browsing large documents , 2000, UIST '00.

[14]  Ulrich Pinkall,et al.  Oorange: A Virtual Laboratory for Experimental Mathematics , 1995, VisMath.

[15]  Takeo Igarashi,et al.  Flatland: new dimensions in office whiteboards , 1999, CHI '99.

[16]  Jeff A. Johnson,et al.  The Xerox Star: a retrospective , 1989, Computer.

[17]  Eric Saund,et al.  Design and technology for Collaborage: collaborative collages of information on physical walls , 1999, UIST '99.

[18]  Jock D. Mackinlay,et al.  Information visualization using 3D interactive animation , 1991, CHI.

[19]  Pat Hanrahan,et al.  A distributed graphics system for large tiled displays , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[20]  Dan Shafer,et al.  The power of PenPoint , 1991 .

[21]  Gordon Kurtenbach,et al.  The design and evaluation of marking menus , 1993 .

[22]  Rick Stevens,et al.  Developing tiled projection display systems , 2000 .

[23]  Rajeev J. Surati.  Scalable self-calibrating display technology for seamless large-scale displays , 1999 .

[24]  Gregory D. Abowd,et al.  Cirrin: a word-level unistroke keyboard for pen input , 1998, UIST '98.

[25]  Axel Kramer,et al.  Translucent patches—dissolving windows , 1994, UIST '94.

[26]  M. Weiser,et al.  An empirical comparison of pie vs. linear menus , 1988, CHI '88.

[27]  Ken Perlin,et al.  Pad: an alternative approach to the computer interface , 1993, SIGGRAPH.

[28]  Jeffrey Johnson,et al.  GUI Bloopers: Don'ts and Do's for Software Developers and Web Designers , 2000 .

[29]  John R. Anderson,et al.  Learning and Memory: An Integrated Approach , 1994 .

[30]  David Goldberg,et al.  Touch-typing with a stylus , 1993, INTERCHI.

[31]  Don Hopkins,et al.  The design and implementation of pie menus , 1991 .

[32]  Thomas P. Moran,et al.  Spatial interpretation of domain objects integrated into a freeform electronic whiteboard , 1998, UIST '98.

[33]  W. Brent Seales,et al.  Multi-projector displays using camera-based registration , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[34]  Pierre David Wellner,et al.  Interacting with paper on the DigitalDesk , 1993, CACM.

[35]  Abigail Sellen,et al.  The Prevention of Mode Errors Through Sensory Feedback , 1992, Hum. Comput. Interact..

[36]  Samuel J. Yen.  Capturing multimodal design activities in support of information retrieval and process analysis , 2000 .

[37]  Jock D. Mackinlay,et al.  The perspective wall: detail and context smoothly integrated , 1991, CHI.

[38]  Jon Trinder,et al.  The Humane Interface: New Directions for Designing Interactive Systems , 2002, Interact. Learn. Environ..

[39]  Asok K. Sen,et al.  Moiré patterns , 2000, Comput. Graph..

[40]  Ben Shneiderman,et al.  Investigating touchscreen typing: the effect of keyboard size on typing speed , 1993, Behav. Inf. Technol..

[41]  James D. Hollan,et al.  Pad++: a zooming graphical interface for exploring alternate interface physics , 1994, UIST '94.

[42]  Daniel G. Bobrow,et al.  WYSIWIS revised: early experiences with multi-user interfaces , 1986, CSCW.

[43]  John C. Tang,et al.  Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration , 1992, CHI.

[44]  Takeo Igarashi,et al.  Designing an Augmented Writing Surface , 2000, IEEE Computer Graphics and Applications.

[45]  Stephen Travis Pope,et al.  A cookbook for using the model-view controller user interface paradigm in Smalltalk-80 , 1988 .

[46]  William Buxton,et al.  Large Displays in Automotive Design , 2000, IEEE Computer Graphics and Applications.

[47]  Xing Chen,et al.  Lumipoint: multi-user laser-based interaction on large tiled displays , 2002 .

[48]  Thomas P. Moran,et al.  Tivoli: an electronic whiteboard for informal workgroup meetings , 1993, INTERCHI.

[49]  Terry Winograd,et al.  Visual instruments for an interactive mural , 1999, CHI Extended Abstracts.

[50]  William Buxton,et al.  The limits of expert performance using hierarchic marking menus , 1993, INTERCHI.

[51]  Maureen C. Stone.  Color and Brightness Appearance Issues in Tiled Displays , 2001, IEEE Computer Graphics and Applications.

[52]  Emmanuel Barillot,et al.  Control menus: excecution and control in a single interactor , 2000, CHI Extended Abstracts.

[53]  Kai Li,et al.  Optical blending for multi-projector display wall systems , 1999, 1999 IEEE LEOS Annual Meeting Conference Proceedings. LEOS'99. 12th Annual Meeting. IEEE Lasers and Electro-Optics Society 1999 Annual Meeting (Cat. No.99CH37009).

[54]  Meredith Ringel Morris,et al.  Barehands: implement-free interaction with a wall-mounted display , 2001, CHI Extended Abstracts.

[55]  John Snyder,et al.  Rendering with coherent layers , 1997, SIGGRAPH.

[56]  Brad A. Myers.  Using multiple devices simultaneously for display and control , 2000, IEEE Wirel. Commun..

[57]  Ken Perlin,et al.  Quikwriting: continuous stylus-based text entry , 1998, UIST '98.

[58]  Jun Rekimoto,et al.  Pick-and-drop: a direct manipulation technique for multiple computer environments , 1997, UIST '97.

[59]  Thomas A. Funkhouser,et al.  Large-format displays , 2000, IEEE Computer Graphics and Applications.

[60]  Norbert A. Streitz,et al.  i-LAND: an interactive landscape for creativity and innovation , 1999, CHI '99.

[61]  Mark A. Jones,et al.  Insight lab: an immersive team environment linking paper, displays, and data , 1998, CHI.

[62]  Eric R. Fossum,et al.  Digital camera system on a chip , 1998, IEEE Micro.

[63]  Nicholas Jackiw,et al.  The geometer's sketchpad: programming by geometry , 1993 .

[64]  Maureen C. Stone.  Color Balancing Experimental Projection Displays , 2001, Color Imaging Conference.

[65]  Marc Levoy,et al.  QSplat: a multiresolution point rendering system for large meshes , 2000, SIGGRAPH.

[66]  Jock D. Mackinlay,et al.  The cognitive coprocessor architecture for interactive user interfaces , 1989, UIST '89.

[67]  Raphael L. Levien.  Highly sensitive register mark based on moire patterns , 1993, Electronic Imaging.

[68]  Andy Hopper,et al.  Virtual Network Computing , 1998, IEEE Internet Comput..

[69]  Benjamin B. Bederson,et al.  Space-scale diagrams: understanding multiscale interfaces , 1995, CHI '95.

[70]  Dan Venolia,et al.  T-Cube: a fast, self-disclosing pen-based alphabet , 1994, CHI '94.

[71]  Pat Hanrahan,et al.  Distributed Rendering for Scalable Displays , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[72]  Takeo Igarashi,et al.  A temporal model for multi-level undo and redo , 2000, UIST '00.

[73]  Begnaud Francis Hildebrand,et al.  Introduction to numerical analysis: 2nd edition , 1987 .

[74]  William Buxton,et al.  Issues in combining marking and direct manipulation techniques , 1991, UIST '91.

[75]  Preston G. Smith.  The art of innovation: lessons in creativity from IDEO, America’s leading design firm: Tom Kelley with Jonathan Littman; New York: Doubleday, 2001, 308 + xii pages, $26.00 , 2002 .

[76]  Mark W. Newman,et al.  The designers' outpost: a tangible interface for collaborative web site , 2001, UIST '01.

[77]  Terry Winograd,et al.  FlowMenu: combining command, text, and data entry , 2000, UIST '00.

[78]  Jun Rekimoto,et al.  A multiple device approach for supporting whiteboard-based interactions , 1998, CHI.