WritLarge: Ink Unleashed by Unified Scope, Action, & Zoom

WritLarge is a freeform canvas for early-stage design on electronic whiteboards with pen+touch input. The system aims to support a higher-level flow of interaction by 'chunking' the traditionally disjoint steps of selection and action into unified selection-action phrases. This holistic goal led us to address two complementary aspects: SELECTION, for which we devise a new technique known as the Zoom-Catcher that integrates pinch-to-zoom and selection in a single gesture for fluidly selecting and acting on content; plus: ACTION, where we demonstrate how this addresses the combined issues of navigating, selecting, and manipulating content. In particular, the designer can transform select ink strokes in flexible and easily-reversible representations via semantic, structural, and temporal axes of movement that are defined as conceptual 'moves' relative to the specified content. This approach dovetails zooming with lightweight specification of scope as well as the evocation of context-appropriate commands, at-hand, in a location-independent manner. This establishes powerful new primitives that can help to scaffold higher-level tasks, thereby unleashing the expressive power of ink in a compelling manner.

[1]  Bill Buxton,et al.  Sketching User Experiences: Getting the Design Right and the Right Design , 2007 .

[2]  Hans-Werner Gellersen,et al.  Partially-indirect Bimanual Input with Gaze, Pen, and Touch for Pan, Zoom, and Ink Interaction , 2016, CHI.

[3]  Ken Perlin,et al.  Pad: an alternative approach to the computer interface , 1993, SIGGRAPH.

[4]  Ben Shneiderman,et al.  Creativity support tools: accelerating discovery and innovation , 2007, CACM.

[5]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[6]  William Buxton,et al.  Tracking menus , 2003, UIST '03.

[7]  George G. Robertson,et al.  Toolspaces and glances: storing, accessing, and retrieving objects in 3D desktop applications , 1999, SI3D.

[8]  George G. Robertson,et al.  Fix and float: object movement by egocentric navigation , 1997, UIST '97.

[9]  Mark W. Newman,et al.  The designers' outpost: a tangible interface for collaborative web site , 2001, UIST '01.

[10]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[11]  Wendy E. Mackay,et al.  BiTouch and BiPad: designing bimanual interaction for hand-held tablets , 2012, CHI.

[12]  Yasuhiro Yamamoto,et al.  Interaction design of tools for fostering creativity in the early stages of information design , 2005, Int. J. Hum. Comput. Stud..

[13]  J. Shah,et al.  Collaborative Sketching (C-Sketch)--An Idea Generation Technique for Engineering Design. , 2001 .

[14]  Mary Czerwinski,et al.  Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-Operated Systems , 2003, INTERACT.

[15]  Robert C. Zeleznik,et al.  Hands-on math: a page-based multi-touch and pen desktop for technical work and problem solving , 2010, UIST.

[16]  Steven J. DeRose,et al.  Document Structure and Markup in the FRESS Hypertext System , 1999, Markup languages.

[17]  Allison Druin,et al.  Technology probes: inspiring design for and with families , 2003, CHI '03.

[18]  Carl Gutwin,et al.  HandMark Menus: Rapid Command Selection and Large Command Sets on Multi-Touch Displays , 2016, CHI.

[19]  Wendy E. Mackay,et al.  Musink: composing music through augmented drawing , 2009, CHI.

[20]  Patrick Baudisch,et al.  The springboard: multiple modes in one spring-loaded control , 2006, CHI.

[21]  Michael Haller,et al.  Harpoon selection: efficient selections for ungrouped content on large pen-based surfaces , 2011, UIST '11.

[22]  Clifton Forlines,et al.  Glimpse: a novel input model for multi-level devices , 2005, CHI EA '05.

[23]  Andrew M. Webb,et al.  Wearables as Context for Guiard-abiding Bimanual Touch , 2016, UIST.

[24]  Daniel J. Wigdor,et al.  Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces , 2008, AVI '08.

[25]  William Buxton,et al.  Issues in combining marking and direct manipulation techniques , 1991, UIST '91.

[26]  William Buxton,et al.  Pen + touch = new tools , 2010, UIST.

[27]  James D. Hollan,et al.  Local tools: an alternative to tool palettes , 1996, UIST '96.

[28]  Frank M. Shipman,et al.  Formality Considered Harmful: Experiences, Emerging Themes, and Directions on the Use of Formal Representations in Interactive Systems , 1999, Computer Supported Cooperative Work (CSCW).

[29]  Donald A. Schön.  Designing as reflective conversation with the materials of a design situation , 1992, Knowl. Based Syst..

[30]  Ted Selker,et al.  Multi-finger chords for hand-held tablets: recognizable and memorable , 2014, CHI.

[31]  Yang Li,et al.  Experimental analysis of mode switching techniques in pen-based user interfaces , 2005, CHI.

[32]  William Buxton,et al.  A marking based interface for collaborative writing , 1993, UIST '93.

[33]  Tony DeRose,et al.  Toolglass and magic lenses: the see-through interface , 1993, SIGGRAPH.

[34]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[35]  C. M. Sperberg-McQueen,et al.  Welcome to Markup Languages: Theory & Practice , 1999, Markup Lang..

[36]  William Buxton,et al.  The design of a GUI paradigm based on tablets, two-hands, and transparency , 1997, CHI.

[37]  Andruid Kerne,et al.  High-performance pen + touch modality interactions: a real-time strategy game eSports context , 2012, UIST.

[38]  Dimitre Novatchev,et al.  Chunking and Phrasing and the Design of Human-Computer Dialogues - Response , 1986, IFIP Congress.

[39]  Edward Lank,et al.  Concurrent bimanual stylus interaction: a study of non-preferred hand mode manipulation , 2006, Graphics Interface.

[40]  George W. Fitzmaurice,et al.  Spotlight: directing users' attention on large displays , 2005, CHI.

[41]  James A. Landay,et al.  NotePals: lightweight note sharing by the group, for the group , 1999, CHI '99.

[42]  Abigail Sellen,et al.  The Prevention of Mode Errors Through Sensory Feedback , 1992, Hum. Comput. Interact..

[43]  D. Hoang.  FLOW: The Psychology of Optimal Experience , 2018 .

[44]  Thomas T. Hewett,et al.  Informing the design of computer-based environments to support creativity , 2005, Int. J. Hum. Comput. Stud..

[45]  Emmanuel Barillot,et al.  Control menus: excecution and control in a single interactor , 2000, CHI Extended Abstracts.

[46]  James D. Foley,et al.  The human factors of computer graphics interaction techniques , 1984, IEEE Computer Graphics and Applications.

[47]  Daniel Vogel,et al.  Pinch-to-zoom-plus: an enhanced pinch-to-zoom that reduces clutching and panning , 2014, UIST.

[48]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[49]  Ian Oakley,et al.  Designing a Physical Aid to Support Active Reading on Tablets , 2015, CHI.

[50]  Jo W. Tombaugh,et al.  Measuring the true cost of command selection: techniques and results , 1990, CHI '90.

[51]  Jon Trinder,et al.  The Humane Interface: New Directions for Designing Interactive Systems , 2002, Interact. Learn. Environ..

[52]  John S. Gero,et al.  Drawings and the design process , 1998 .

[53]  Fabrice Matulic,et al.  Pen and touch gestural environment for document editing on interactive tabletops , 2013, ITS.

[54]  Takeo Igarashi,et al.  Flatland: new dimensions in office whiteboards , 1999, CHI '99.

[55]  Edward Lank,et al.  A study on the scalability of non-preferred hand mode manipulation , 2007, ICMI '07.

[56]  Sachi Mizobuchi,et al.  Tapping vs. circling selections on pen-based devices: evidence for different performance-shaping factors , 2004, CHI '04.

[57]  David Lindlbauer,et al.  Perceptual grouping: selection assistance for digital sketching , 2013, ITS.

[58]  Patrick Baudisch,et al.  Design and analysis of delimiters for selection-action pen gesture phrases in scriboli , 2005, CHI.

[59]  Xiaojun Bi,et al.  Informal information gathering techniques for active reading , 2012, CHI.

[60]  Thomas P. Moran,et al.  Pen-based interaction techniques for organizing material on an electronic whiteboard , 1997, UIST '97.

[61]  Michael Haller,et al.  cLuster: Smart Clustering of Free-Hand Sketches on Large Interactive Surfaces , 2015, UIST.

[62]  Axel Kramer,et al.  Translucent patches—dissolving windows , 1994, UIST '94.

[63]  Myron W. Krueger,et al.  VIDEOPLACE—an artificial reality , 1985, CHI '85.

[64]  Scott E. Hudson,et al.  A framework for robust and flexible handling of inputs with uncertainty , 2010, UIST.

[65]  Patrick Baudisch,et al.  Hover widgets: using the tracking state to extend the capabilities of pen-operated devices , 2006, CHI.

[66]  Anastasia Bezerianos,et al.  The vacuum: facilitating the manipulation of distant objects , 2005, CHI.

[67]  Ravin Balakrishnan,et al.  Keepin' it real: pushing the desktop metaphor with physics, piles and the pen , 2006, CHI.

[68]  Brian P. Bailey,et al.  TEAM STORM: demonstrating an interaction model for working with multiple ideas during creative group work , 2007, C&C '07.