Beyond the Digital Capture of Paper Notes: Investigations of Enhanced Natural Notetaking based on Digital Pen and Paper Technology

Even if digital approaches are often considered better than paper for the handling and management of information, research has shown that a paperless office is far from being achieved. Some practices are interwoven with a paper-based information capture—or, in other words, paper-based notetaking—in ways difficult to reproduce in digital systems. Paper is the medium preferred to capture information especially in situations characterised by aspects such as spontaneity or a certain need to be flexible and unconstrained. For example, writing with a pen on a piece of paper is often the most straightforward approach for offloading details from mind in preparation for a different activity. Throughout this thesis, we refer to the aforementioned and similar situations as natural notetaking. Anoto’s digital pen and paper technology and similar paper-digital solutions create the possibility to bring together the preference for using paper as capture medium with the benefits of digital information handling. With the aid of these technologies, handwritten information captured on paper can be made available digitally without an intermediary transcription step. Nevertheless, even though Anoto and similar technologies have now been around for approximately a decade, their use for everyday natural notetaking is limited. This thesis investigates possible reasons for this situation and makes a series of proposals towards aligning the use of Anoto’s digital pen and paper technology with natural notetaking. Our investigations start with a study on natural notetaking practices. The study reveals two main areas to be investigated: support for recall of valuable handwritten notes and providing overviews of the use of paper notes in combination with other types of media. The latter is particularly relevant in collaborative settings. The study also shows a general reluctance of the users towards solutions that would require the adaptation of natural notetaking practices. At the same time, the study shows that some mechanism to support the processing of notes is necessary. Existing solutions based on digital pen and paper rely on the use of certain rules or guidelines to enforce the processing of information captured on paper, such as requiring users to write within special purpose page areas or use predefined marking gestures. These user-driven processing approaches serve as support for deriving metadata based on which the organisation of notes in digital space is inferred. In an attempt to deal with potential limitations of user-driven processing on natural notetaking, we designed and implemented an extensible digital ink segmentation and classification framework. The framework supports the development of a wide variety of automatic approaches for processing the digital ink representations of paper-based notes, as well as the combination of automatic and user-driven processing approaches. The framework is meant to be used on top of existing frameworks and toolkits for developing digital pen and paper-based applications, with which it can be easily integrated. By allowing the easy