Pen and touch gestural environment for document editing on interactive tabletops

Combined pen and touch input is an interaction paradigm attracting increasing interest both in the research community and recently in industry. In this paper, we illustrate how pen and touch interaction techniques can be leveraged for editing and authoring of presentational documents on digital tabletops. Our system exploits the rich interactional vocabulary afforded by the simultaneous availability of the two modalities to provide gesture-driven document editing functionality as an expert alternative to widgets. For our bimanual gestures, we make use of non-dominant hand postures to set pen modes in which the dominant hand articulates a variety of transactions. We draw an analogy between such modifier postures and modifier keys on a keyboard to construct command shortcuts. Based on this model, we implement a number of common document editing operations, including several page and element manipulations, shape and text input with styling, clipart retrieval and insertion as well as undo/redo. The results of a lab study provide in-sights as to the strengths and limitations of our approach.