Enhancing Multimodal Annotations with Pen-Based Information

Ubiquitous computing aims at providing services to users without taking their attention from their main task. Considering that, when users watch a video with someone else, they are used to make comments with respect to its contents, we propose the capture of typical video watching- and-commenting tasks to automatically generate a corre- sponding annotated interactive video. In this project we first revisit our previous experiences with multimodal annotations and ink playback. Next, we discuss our work in progress toward building a prototype that supports the authoring by watching-and-commenting paradigm, in which users can add audio or pen-based com- ments to digital video or to a sequence of images.