As We May Ink?: Learning from Everyday Analog Pen Use to Improve Digital Ink Experiences

This paper sheds light on gaps and discrepancies between the experiences afforded by analog pens and their digital counterparts. Despite the long history (and recent renaissance) of digital pens, the literature still lacks a comprehensive survey of what types of marks people make and what motivates them to use ink-both analog and digital in daily life. To capture the diversity of inking behaviors and tease out the unique affordances of pen-and ink, we conducted a diary study with 26 participants from diverse backgrounds. From analysis of 493 diary entries we identified 8 analog pen-and-ink activities, and 9 affordances of pens. We contextualized and contrasted these findings using a survey with 1,633 respondents and a follow-up diary study with 30 participants, observing digital pens. Our analysis reveals gaps and research opportunities based on pen affordances not yet fully explored in the literature.

[1]  James A. Landay,et al.  NotePals: lightweight note sharing by the group, for the group , 1999, CHI '99.

[2]  Abigail Sellen,et al.  The Prevention of Mode Errors Through Sensory Feedback , 1992, Hum. Comput. Interact..

[3]  Barry Arons,et al.  The audio notebook: paper and pen interaction with structured speech , 2001, CHI.

[4]  Benjamin B. Bederson,et al.  Interfaces for staying in the flow , 2004, UBIQ.

[5]  John Bennett,et al.  Reflective conversation with materials , 1996 .

[6]  W. Keith Edwards,et al.  Active reading and its discontents: the situations, problems and ideas of readers , 2011, CHI.

[7]  Anoop Gupta,et al.  The pen is mightier: understanding stylus behaviour while inking on tablets , 2014, Graphics Interface.

[8]  Saul Greenberg,et al.  Cross-device interaction via micro-mobility and f-formations , 2012, UIST.

[9]  Maneesh Agrawala,et al.  FingerGlass: efficient multiscale interaction on multitouch screens , 2011, CHI.

[10]  David Kirsh,et al.  Complementary Strategies: Why we use our hands when we think , 1995 .

[11]  David Jones,et al.  Discerning structure from freeform handwritten notes , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[12]  M. Angela Sasse,et al.  "Stuff goes into the computer and doesn't come out": a cross-tool study of personal information management , 2004, CHI.

[13]  James D. Hollan,et al.  Papiercraft: A gesture-based command system for interactive paper , 2008, TCHI.

[14]  Yang Li,et al.  Experimental analysis of mode switching techniques in pen-based user interfaces , 2005, CHI.

[15]  Ming Ye,et al.  Learning to Group Text Lines and Regions in Freeform Handwritten Notes , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[16]  Thomas T. Hewett,et al.  Informing the design of computer-based environments to support creativity , 2005, Int. J. Hum. Comput. Stud..

[17]  Kenton O'Hara,et al.  A diary study of work-related reading: design implications for digital reading devices , 1998, CHI.

[18]  Bill N. Schilit,et al.  From reading to retrieval: freeform ink annotations as queries , 1999, SIGIR '99.

[19]  Pourang Irani,et al.  Sensing Tablet Grasp + Micro-mobility for Active Reading , 2015, UIST.

[20]  Nadir Weibel,et al.  Digital pen and paper practices in observational research , 2012, CHI.

[21]  Nicholas Chen,et al.  Designing a multi-slate reading environment to support active reading activities , 2012, TCHI.

[22]  M. Sheelagh T. Carpendale,et al.  Visual Thinking In Action: Visualizations As Used On Whiteboards , 2011, IEEE Transactions on Visualization and Computer Graphics.

[23]  Abigail Sellen,et al.  Affordances for manipulation of physical versus digital media on interactive surfaces , 2007, CHI.

[24]  Robert DeLine,et al.  Let's go to the whiteboard: how and why software developers use drawings , 2007, CHI.

[25]  F. R. Wilson.  The Hand: How Its Use Shapes the Brain, Language, and Human Culture , 1998 .

[26]  François Coldefy,et al.  Sketched menus and iconic gestures, techniques designed in the context of shareable interfaces , 2010, ITS '10.

[27]  Wayne G. Lutters,et al.  Understanding the micronote lifecycle: improving mobile support for informal note taking , 2004, CHI.

[28]  Bill N. Schilit,et al.  XLibris: the active reading machine , 1998, CHI Conference Summary.

[29]  Michael S. Bernstein,et al.  Information ! Scraps : ! ! How ! and ! Why ! Information ! Eludes ! our ! Personal ! Information ! Management , 2022 .

[30]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[31]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[32]  Christian Heath,et al.  Mobility in collaboration , 1998, CSCW '98.

[33]  Bill N. Schilit,et al.  Beyond paper: supporting active reading with free form digital ink annotations , 1998, CHI.

[34]  Kenneth R. Koedinger,et al.  Selection-based note-taking applications , 2007, CHI.

[35]  Christine Reid,et al.  The Myth of the Paperless Office , 2003, J. Documentation.

[36]  Ian Oakley,et al.  Designing a Physical Aid to Support Active Reading on Tablets , 2015, CHI.

[37]  Tony DeRose,et al.  Determining the benefits of direct-touch, bimanual, and multifinger input on a multitouch workstation , 2009, Graphics Interface.

[38]  Kenton O'Hara,et al.  A comparison of reading paper and on-line documents , 1997, CHI.

[39]  Jun Rekimoto,et al.  Z-touch: an infrastructure for 3d gesture interaction in the proximity of tabletop surfaces , 2010, ITS '10.

[40]  Beryl Plimmer,et al.  Freeform digital ink annotations in electronic documents: A systematic mapping study , 2016, Comput. Graph..

[41]  Meredith Ringel Morris,et al.  Reading Revisited: Evaluating the Usability of Digital Display Surfaces for Active Reading Tasks , 2007, Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP'07).

[42]  Catherine C. Marshall,et al.  Turning the page on navigation , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[43]  Peter Brandl,et al.  NiCEBook: supporting natural note taking , 2010, CHI.

[44]  J. Shah,et al.  Collaborative Sketching (C-Sketch)--An Idea Generation Technique for Engineering Design. , 2001 .

[45]  Catherine C. Marshall,et al.  Annotation: from paper books to the digital library , 1997, DL '97.

[46]  Nicholas Chen,et al.  TextTearing: opening white space for digital ink annotation , 2013, UIST.

[47]  William Buxton,et al.  Inking Outside the Box: How Context Sensing Affords More Natural Pen (and Touch) Computing , 2016, Revolutionizing Education with Digital Ink.

[48]  Alison Kidd,et al.  The marks are on the knowledge worker , 1994, CHI '94.

[49]  Mark W. Newman,et al.  The designers' outpost: a tangible interface for collaborative web site , 2001, UIST '01.

[50]  Shun'ichi Tano,et al.  Microscopic analysis of document handling while reading paper documents to improve digital reading device , 2014, OZCHI.

[51]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[52]  Catherine C. Marshall,et al.  Exploring the relationship between personal and public annotations , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[53]  Hirohito Shibata,et al.  Effects of Paper on Page Turning: Comparison of Paper and Electronic Media in Reading Documents with Endnotes , 2011, HCI.

[54]  W. Keith Edwards,et al.  LiquidText: a flexible, multitouch environment to support active reading , 2011, CHI.

[55]  Anoop Gupta,et al.  Exploring and Understanding Unintended Touch during Direct Pen Interaction , 2014, TCHI.

[56]  Daniel M. Oppenheimer,et al.  The Pen Is Mightier Than the Keyboard , 2014, Psychological science.

[57]  Anne Marie Piper,et al.  Microanalysis of active reading behavior to inform design of interactive desktop workspaces , 2012, ITS.

[58]  Christine L. MacKenzie,et al.  The Grasping Hand , 2011, The Grasping Hand.

[59]  Xiaojun Bi,et al.  Informal information gathering techniques for active reading , 2012, CHI.

[60]  Thomas P. Moran,et al.  Pen-based interaction techniques for organizing material on an electronic whiteboard , 1997, UIST '97.