Tilt-Responsive Techniques for Digital Drawing Boards

Drawing boards offer a self-stable work surface that is continuously adjustable. On digital displays, such as the Microsoft Surface Studio, these properties open up a class of techniques that sense and respond to tilt adjustments. Each display posture-whether angled high, low, or somewhere in-between-affords some activities, but not others. Because what is appropriate also depends on the application and task, we explore a range of app-specific transitions between reading vs. writing (annotation), public vs. personal, shared person-space vs. task-space, and other nuances of input and feedback, contingent on display angle. Continuous responses provide interactive transitions tailored to each use-case. We show how a variety of knowledge work scenarios can use sensed display adjustments to drive context-appropriate transitions, as well as technical software details of how to best realize these concepts. A preliminary remote user study suggests that techniques must balance effort required to adjust tilt, versus the potential benefits of a sensed transition.

[1]  Hiroo Iwata,et al.  Project FEELEX: adding haptic surface to graphics , 2001, SIGGRAPH.

[2]  Hiroshi Ishii,et al.  inFORCE: Bi-directional `Force' Shape Display for Haptic Interaction , 2019, TEI.

[3]  Nicolai Marquardt,et al.  Proxemic interactions: the new ubicomp? , 2011, INTR.

[4]  Yoshifumi Kitamura,et al.  TransformTable: a self-actuated shape-changing digital table , 2013, ITS.

[5]  Johannes Schwank,et al.  Waggle -- Orientation-based Tablet Interaction , 2017, CHI Extended Abstracts.

[6]  William Buxton,et al.  Pen + touch = new tools , 2010, UIST.

[7]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[8]  Jens Emil Grønbæk,et al.  Proxemic Transitions: Designing Shape-Changing Furniture for Informal Meetings , 2017, CHI.

[9]  Kenton O'Hara,et al.  Sensitizing Concepts for Socio-spatial Literacy in HCI , 2017, CHI.

[10]  Wendy Ju,et al.  The Design of Implicit Interactions , 2015, Synthesis Lectures on Human-Centered Informatics.

[11]  Meredith Ringel Morris,et al.  Reading Revisited: Evaluating the Usability of Digital Display Surfaces for Active Reading Tasks , 2007, Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07).

[12]  Ehud Sharlin,et al.  Study and Design of a Shape-Shifting Wall Display , 2016, Conference on Designing Interactive Systems.

[13]  Ben Shneiderman,et al.  High Precision Touchscreens: Design Strategies and Comparisons with a Mouse , 1991, Int. J. Man Mach. Stud..

[14]  Donald A. Schön.  Designing as reflective conversation with the materials of a design situation , 1992, Knowl. Based Syst..

[15]  Jonathan Grudin,et al.  Integration of interpersonal space and shared workspace: ClearBoard design and experiments , 1993, TOIS.

[16]  Jan O. Borchers,et al.  BendDesk: dragging across the curve , 2010, ITS '10.

[17]  Eric Horvitz,et al.  Foreground and background interaction with sensor-enhanced mobile devices , 2005, TCHI.

[18]  Michel Beaudouin-Lafon,et al.  SPad: a bimanual interaction technique for productivity applications on multi-touch tablets , 2014, CHI Extended Abstracts.

[19]  Raimund Dachselt,et al.  Foldable3D: Interacting with 3D Content Using Dual-Display Devices , 2016, ISS.

[20]  Meredith Ringel Morris,et al.  A field study of knowledge workers’ use of interactive horizontal displays , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[21]  Jun Rekimoto,et al.  Tilting operations for small screen interfaces , 1996, UIST '96.

[22]  Mary Czerwinski,et al.  Data mountain: using spatial memory for document management , 1998, UIST '98.

[23]  Xiang Cao,et al.  Enhancing naturalness of pen-and-tablet drawing through context sensing , 2011, ITS '11.

[24]  P. Fitts,et al.  S-R compatibility: spatial characteristics of stimulus and response codes. , 1953, Journal of experimental psychology.

[25]  Hemant Bhaskar Surale,et al.  TabletInVR: Exploring the Design Space for Using a Multi-Touch Tablet in Virtual Reality , 2019, CHI.

[26]  Kasper Hornbæk,et al.  An experimental comparison of touch interaction on vertical and horizontal surfaces , 2012, NordiCHI.

[27]  S. Hudson,et al.  CHI '08 Extended Abstracts on Human Factors in Computing Systems , 2009, CHI 2009.

[28]  Andrew Sears,et al.  A new era for high precision touchscreens , 1993 .

[29]  Peter Brandl,et al.  Flux: a tilting multi-touch and pen based surface , 2009, CHI Extended Abstracts.

[30]  Ravin Balakrishnan,et al.  Storeoboard: Sketching Stereoscopic Storyboards , 2016, CHI.

[31]  Michael S. Bernstein,et al.  Information ! Scraps : ! ! How ! and ! Why ! Information ! Eludes ! our ! Personal ! Information ! Management , 2022 .

[32]  Ian Taylor,et al.  Designing novel interactional workspaces to support face to face consultations , 2003, CHI '03.

[33]  Yoshihiro Kawahara,et al.  Dynablock: Dynamic 3D Printing for Instant and Reconstructable Shape Formation , 2018, UIST.

[34]  Roy Want,et al.  Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces , 1998, CHI.

[35]  Adam Kendon,et al.  Spacing and Orientation in Co-present Interaction , 2009, COST 2102 Training School.

[36]  Wendy E. Mackay,et al.  BiTouch and BiPad: designing bimanual interaction for hand-held tablets , 2012, CHI.

[37]  Paul Dourish,et al.  Re-place-ing space: the roles of place and space in collaborative systems , 1996, CSCW '96.

[38]  R. Belk.  Possessions and the Extended Self , 1988 .

[39]  Kasper Hornbæk,et al.  Technology Acceptance and User Experience , 2017, ACM Trans. Comput. Hum. Interact..

[40]  Darren Leigh,et al.  DiamondTouch: a multi-user touch technology , 2001, UIST '01.

[41]  Jun Rekimoto,et al.  Lumen: interactive visual and shape display for calm computing , 2004, SIGGRAPH '04.

[42]  Roman Rädle,et al.  Cross-Device Taxonomy: Survey, Opportunities and Challenges of Interactions Spanning Across Multiple Devices , 2019, CHI.

[43]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[44]  Norbert A. Streitz,et al.  Connectables: dynamic coupling of displays for the flexible creation of shared workspaces , 2001, UIST '01.

[45]  Yann Riche,et al.  As We May Ink?: Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences , 2017, CHI.

[46]  Christine Reid,et al.  The Myth of the Paperless Office , 2003, J. Documentation.

[47]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[48]  Hiroshi Ishii,et al.  ClearBoard: a seamless medium for shared drawing and conversation with eye contact , 1992, CHI.

[49]  Saul Greenberg,et al.  Cross-device interaction via micro-mobility and f-formations , 2012, UIST.

[50]  Andreas Butz,et al.  Curve: revisiting the digital desk , 2010, NordiCHI.

[51]  E. Hall,et al.  The Hidden Dimension , 1970 .

[52]  William Buxton,et al.  Thumb + Pen Interaction on Tablets , 2017, CHI.

[53]  K CardStuart,et al.  A morphological analysis of the design space of input devices , 1991 .

[54]  Ken Hinckley,et al.  Sensor synaesthesia: touch in motion, and motion in touch , 2011, CHI.

[55]  J. Bartlett.  WRL Research Report 2000 / 3 Rock ’ n ’ Scroll is Here to Stay , 2000 .

[56]  William Buxton,et al.  Lexical and pragmatic considerations of input structures , 1983, COMG.

[57]  Fei Su,et al.  Sensing Posture-Aware Pen+Touch Interaction on Tablets , 2019, CHI.

[58]  Hyun-Jean Lee,et al.  Tilting table: a movable screen , 2007, Tangible and Embedded Interaction.

[59]  Regan L. Mandryk,et al.  System Guidelines for Co-located, Collaborative Work on a Tabletop Display , 2003, ECSCW.

[60]  Stuart K. Card,et al.  Powermice and user performance , 1990, CHI '90.

[61]  William Buxton,et al.  Large Displays in Automotive Design , 2000, IEEE Computer Graphics and Applications.

[62]  William Buxton,et al.  Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display , 2002, UIST '02.

[63]  Daniel Vogel,et al.  HybridPointing: fluid switching between absolute and relative pointing with a direct input device , 2006, UIST.

[64]  Christian Heath,et al.  Mobility in collaboration , 1998, CSCW '98.

[65]  Kenton O'Hara,et al.  A diary study of work-related reading: design implications for digital reading devices , 1998, CHI.

[66]  Thomas A. DeFanti,et al.  Varrier autostereographic display , 2001, IS&T/SPIE Electronic Imaging.

[67]  Y. Guiard,et al.  Writing postures in left-handers: Inverters are hand-crossers , 1984, Neuropsychologia.

[68]  Hiroshi Ishii,et al.  The metaDESK: models and prototypes for tangible user interfaces , 1997, UIST '97.

[69]  Hideaki Kuzuoka,et al.  Handling documents and discriminating objects in hybrid spaces , 2006, CHI.

[70]  Raimund Dachselt,et al.  Throw and Tilt - Seamless Interaction across Devices Using Mobile Phone Gestures , 2008, GI Jahrestagung.

[71]  Eric Lecolinet,et al.  TimeTilt: Using Sensor-Based Gestures to Travel through Multiple Applications on a Mobile Device , 2009, INTERACT.

[72]  John C. Tang,et al.  Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration , 1992, CHI.

[73]  Jens Emil Grønbæk,et al.  Proxemics Beyond Proximity: Designing for Flexible Social Interaction Through Cross-Device Interaction , 2020, CHI.

[74]  Terry Winograd,et al.  Fluid interaction with high-resolution wall-size displays , 2001, UIST '01.

[75]  Ravin Balakrishnan,et al.  Keepin' it real: pushing the desktop metaphor with physics, piles and the pen , 2006, CHI.

[76]  Shumin Zhai,et al.  Virtual reality for palmtop computers , 1993, TOIS.

[77]  Nicholas Chen,et al.  TextTearing: opening white space for digital ink annotation , 2013, UIST.

[78]  Alison Kidd,et al.  The marks are on the knowledge worker , 1994, CHI '94.

[79]  Ravin Balakrishnan,et al.  Codex: a dual screen tablet computer , 2009, CHI.

[80]  Pernilla Qvarfordt,et al.  Optimal Orientation of Text Documents for Reading and Writing , 2020, Hum. Comput. Interact..

[81]  D. Ariely,et al.  The 'IKEA Effect': When Labor Leads to Love , 2011 .

[82]  William Buxton,et al.  Telepresence: Integrating Shared Task and Person Spaces , 1992 .

[83]  Andy Cockburn,et al.  Touch scrolling transfer functions , 2013, UIST.

[84]  Abigail Sellen,et al.  Lost in translation: understanding the possession of digital things in the cloud , 2012, CHI.

[85]  Maxine D. Brown,et al.  The ImmersaDesk and Infinity Wall projection-based virtual reality displays , 1997, COMG.

[86]  Christian Müller-Tomfelde,et al.  Tilted tabletops: In between horizontal and vertical workspaces , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[87]  Michael Boyle,et al.  PDAs and shared public displays: Making personal information public, and public information personal , 1999, Personal Technologies.

[88]  Carl Gutwin,et al.  Improving Discoverability and Expert Performance in Force-Sensitive Text Selection for Touch Devices with Mode Gauges , 2018, CHI.

[89]  Sebastian Boring,et al.  Hover Pad: interacting with autonomous and self-actuated displays in space , 2014, UIST.

[90]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[91]  Gerald Penn,et al.  Living with a Tabletop: Analysis and Observations of Long Term Office Use of a Multi-Touch Table , 2007, Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07).

[92]  Jonathan Grudin,et al.  Partitioning digital worlds: focal and peripheral awareness in multiple monitor use , 2001, CHI.

[93]  Jock D. Mackinlay,et al.  A morphological analysis of the design space of input devices , 1991, TOIS.

[94]  Colin Ware,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990, I3D '90.

[95]  C. M. Deasy,et al.  Designing Places for People: "A Handbook on Human Behavior for Architects, Designers and Facility Managers" , 1985 .